PaperlayerView exact revision on arXiv

Source-linked AI summary

MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs

Zimu Lu, Aojun Zhou, Houxing Ren, Ke Wang, Weikang Shi, Junting Pan, Mingjie Zhan, Hongsheng Li

arXiv:2402.16352v2cs.CLcs.AI

TL;DR

MathGenie tackles the cost of GPT-4-based data scaling and the challenge of creating diverse math problems with reliable solutions. It augments solutions, back-translates them into questions, and filters code-integrated answers; MathGenieLM outperforms previous open-source models across five datasets, with leading reported results on GSM8K and MATH.

  • Problem

    GPT-4-based generation makes scaling math training data prohibitively expensive, and generating diverse problems with accurate solutions without ground truth remains a challenge.

    Source: §1 · pp. 1–2

  • Method

    MathGenie augments seed solutions, back-translates them into new questions, then uses verification rationales to filter generated code-integrated solutions.

    Source: §1 · p. 1, §2 · p. 4

  • Results

    87.7% on GSM8K and 55.7% on MATH for MathGenieLM-InternLM2, the best overall score among open-source models, within results spanning five datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The coordinated pipeline creates new math questions and filters their code-integrated solutions for reliability.

    Source: §5 · p. 8

    The approach requires significant GPU resources for full-parameter fine-tuning of models up to 70B parameters.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language models (LLMs) have exhibited great potential in mathematical reasoning. However, there remains a performance gap in this area between existing open-source models and closed-source models such as GPT-4. In this paper, we introduce MathGenie, a novel method for generating diverse and reliable math problems from a small-scale problem-solution dataset (denoted as seed data). We augment the ground-truth solutions of our seed data and train a back-translation model to translate the augmented solutions back into new questions. Subsequently, we generate code-integrated solutions for the new questions. To ensure the correctness of the code-integrated solutions, we employ rationale-based strategy for solution verification. Various pretrained models, ranging from 7B to 70B, are trained on the newly curated data to test the effectiveness of the proposed augmentation technique, resulting in a family of models known as MathGenieLM. These models consistently outperform previous open-source models across five representative mathematical reasoning datasets, achieving state-of-the-art performance. In particular, MathGenieLM-InternLM2 achieves an accuracy of 87.7% on GSM8K and 55.7% on MATH, securing the best overall score among open-source language models.

1 Introduction

MathGenie addresses the cost of GPT-4-based data scaling and the challenge of generating diverse problems with reliable solutions. Its solution-first augmentation and verification pipeline produces MathGenieData, while MathGenieLM performs strongly across five math datasets.

1 Introduction

  • GPT-4-based data scaling is prohibitively expensive, while synthetic-data generation must produce diverse problems and reliable solutions without ground truth.MathGenie frames these as two challenges: generating high-quality, diverse problems and accurate solutions, preferably in code-integrated form.Source: §1 · pp. 1–2
  • MathGenie augments human-annotated solutions from MATH and GSM8K, then back-translates them into questions to preserve solution constraints and solvability.The approach uses augmented solutions as the source for question generation rather than directly modifying questions.Source: §1 · pp. 1–2
  • For generated questions without ground-truth solutions, MathGenie generates code-integrated answers and retains only those verified through natural-language-and-code rationales.The verification rationales are used to check the correctness of model-generated solutions.Source: §1 · p. 2
  • Based on the proposed MathGenie framework, researchers obtain MathGenieData, a large-scale model-generated math problem-solution dataset featuring diverse augmented questions and reliable code-integrated solutions.Source: §1 · p. 2
  • Using MathGenie’s question-solution augmentation framework, finetuned MathGenieLM-InternLM2 achieves 87.7% accuracy on GSM8K and 55.7% on MATH, attaining the best overall score.MathGenieLM demonstrates high accuracy across five diverse, representative math datasets: MATH, GSM8K, SVAMP, Simuleq, and Mathematics.Source: §1 · p. 2

2 MathGenie

MathGenie builds a pipeline that augments solutions from seed data, back-translates them into questions, and filters generated code-integrated solutions with verification rationales. The curated question-solution pairs form MathGenieData for fine-tuning mathematical reasoning models.

Seed Data

  • The augmentation seed contains 15K GSM8K and MATH problems with human-annotated solutions, while Dcode contains 80K code-integrated solutions and verification rationales.Dtext is used for question augmentation; Dcode trains the candidate-solution generator.Source: §2 · p. 3

Iterative Solution Augmentation

  • Iterative augmentation creates successive, increasingly different solution sets, which the authors say supports diverse question-solution pairs and benefits final performance.The final augmented set is the union of solution sets from rounds 1 through K.Source: §2 · p. 4

Question Back-translation

  • A model fine-tuned on reversed question-solution pairs translates augmented solutions into new questions, using solution constraints to support more reliable generation than direct question augmentation.The input is a solution and the output is its corresponding question.Source: §2 · p. 4

Verification-based Solution Filtering

  • Mcode is trained to produce interleaved natural-language-and-code verification rationales, then checks generated solutions after answer-consistency filtering.Questions whose candidate solutions produce different answers are removed first; solutions verified as wrong are discarded.Source: §2 · pp. 4–5

MathGenie Data

  • The pipeline produces 170K augmented question and code-integrated solution pairs, combined with SeedData as MathGenieData for fine-tuning models in the MathGenieLM family.AugData contains 110K GSM8K-derived and 60K MATH-derived samples.Source: §2 · p. 5

3 Experiments

MathGenieLM achieves state-of-the-art performance among open-source models across five math datasets, while majority voting and ablations examine the effects of inference and data choices. The experiments also assess generalization, verification filtering, and verified inference.

3.1 Experimental Setup

  • The evaluation finetunes models from 7B to 70B parameters and compares them with open- and closed-source systems across two in-domain and three out-of-domain datasets.The out-of-domain datasets are used to assess generalization.Source: §3.1 · p. 5

3.2 Main Results

  • MathGenieLM achieves state-of-the-art performance among open-source models across five datasets, performs particularly well out of domain, and remains below GPT-4.The evaluation covers two in-domain and three out-of-domain datasets; results use greedy decoding.Source: §3.2 · p. 5, Table 3 · p. 6
  • At k = 10, majority voting yields a 7.9% average gain, with 91.5% on GSM8K and 63.3% on MATH, outperforming ToRA-70B at k = 50.The reported gains cover all five datasets.Source: §3.2 · p. 5, Table 4 · p. 6

3.3 Ablation Study

  • Adding AugGSM8K improves GSM8K, SVAMP, and Simuleq, while AugMATH brings more notable gains on MATH and Mathematics; combining them compounds improvements.The first group contains relatively easier grade-school word problems, while the latter benchmarks involve more complex computations.Source: §3.3 · p. 6, Table 5 · p. 7
  • Increasing augmented-data amounts improves performance on all five datasets, with very few exceptions.The tested amounts range from none to the full generated amount.Source: Table 5 · p. 7, §3.3 · p. 7
  • Verification-based filtering produces noticeable accuracy increases on GSM8K and MATH compared with training on unfiltered augmented pairs.The comparison tests augmented question-solution pairs before and after filtering by verification results.Source: Table 6 · p. 7, §3.3 · p. 7
  • Solution-to-question back-translation performs better than MetaMath and direct question augmentation with or without solutions.The direct augmentation variants use Mtext; one receives seed questions only and the other question-solution pairs.Source: §3.3 · p. 7, Table 7 · p. 8

3.4 Accuracy of Verified Inference

  • Verified inference, repeated twice, improves accuracy across all five datasets and achieves nearly the same accuracy as 3-path majority voting at lower cost.Its average generation cost is 2.3×; improvements are notable on MATH and Mathematics.Source: §3.4 · pp. 7–8, Table 8 · p. 8

4 Related Works

Related work covers mathematical reasoning models and synthetic instruction data, alongside approaches for augmenting math problems. MathGenie instead uses solution information to back-translate questions and filters code-integrated solutions with verification rationales.

Large Language Models for Mathematical Reasoning

  • Prior mathematical reasoning work spans CoT, Self-Consistency, code-based self-verification, and math-specialized models, with code-integrated models reported to outperform CoT-style models.The paper also distinguishes its focus on math-specific code-integrated models from synthetic instruction work for general chat models.Source: §4 · p. 8

Data Augmentation for Mathematical Reasoning

  • Unlike direct question augmentation, MathGenie uses information in solutions for question back-translation, then creates code-integrated solutions and filters them using verification rationales.The related-work passage contrasts this focus with prior math-problem augmentation approaches.Source: §4 · p. 8

Instruction-following Datasets for LLMs

  • Synthetic instructions generated by GPT-4 or GPT-3.5 have been used to distill into smaller models, while MathGenie focuses on augmenting math problems.WizardLM enriches seed data with complex instructions for general chat models.Source: §4 · p. 8

5 Conclusion

MathGenie combines question back-translation with verification-based filtering to produce synthetic math questions and code-integrated solutions. Experiments report superior performance across five math benchmarks and six pretrained base models.

5 Conclusion

  • The pipeline combines iterative solution augmentation and question back-translation to generate synthetic math questions, then filters generated code-integrated solutions through verification.The conclusion presents the three components as a coordinated pipeline.Source: §5 · p. 8
  • The authors present the pipeline as a way to create new questions and ensure the reliability of corresponding code-integrated solutions.This is the stated purpose of combining the three pipeline components.Source: §5 · p. 8
  • MathGenieLM achieves superior performance across five math problem-solving benchmarks and six pretrained base models.The conclusion describes the results as offering insights for math problem-solving models and potential extension to other reasoning tasks.Source: §5 · pp. 8–9

Limitations

The paper identifies resource demands, image-input limitations, and context length as constraints on its models.

Limitations

  • Full-parameter finetuning of models up to 70B requires significant GPU resources, motivating future work to reduce resource requirements.The authors identify resource reduction as an area for future study.Source: Limitations · p. 9
  • The models cannot process images and therefore lack the ability to solve problems involving images.The paper cites Lu et al. (2023) in discussing this limitation.Source: Limitations · p. 9
  • The models are constrained by a context length of 4096 from finetuning.The authors state that these limitations merit further investigation.Source: Limitations · p. 9

Ethics Statement

The paper notes both the potential educational contribution of stronger mathematical abilities and the possibility of hallucinated outputs. It also describes iterative solution augmentation and back-translation as producing more diverse question phrasing than direct question augmentation.

Ethics Statement

  • The models may still output untrue hallucinations, despite their potential contribution to math education.Source: Ethics Statement · p. 9

A Example of Iterative Solution Augmentation and Question Back-translation

B Analysis of Iterative Solution Augmentation

The analysis reports that solution augmentation without iteration performs significantly worse than iterative augmentation, with increased solution diversity offered as a possible explanation. It also identifies the prompt format used for fine-tuning and generating verification rationales.

B Analysis of Iterative Solution Augmentation

  • Solution augmentation without iteration produced significantly inferior outcomes to iterative augmentation.The authors attribute iteration's benefit primarily to its potential to enhance solution diversity.Source: B Analysis of Iterative Solution… · p. 11
  • The comparison generated equal numbers of augmented solutions with and without iteration, using direct augmentation from the original set as the non-iterative condition.Source: B Analysis of Iterative Solution… · p. 11

C Prompt of Verification

D Analysis of Code-integrated Verification

The analysis evaluates code-integrated verification against ground-truth correctness and reports higher precision than accuracy across all five datasets. It defines precision and recall as measures of verification reliability and filtering efficiency, respectively.

D Analysis of Code-integrated Verification

  • The evaluation tests Mcode's code-integrated verification on solutions it generated across five datasets, whose ground truth permits assessment of actual correctness.Source: D Analysis of Code-integrated… · p. 11
  • The analysis defines Precision and Recall to demonstrate Mcode's ability to verify its solutions.Source: D Analysis of Code-integrated… · p. 11
  • Precision measures the proportion of verified-TRUE answers that are actually correct, while Recall measures the proportion of actual correct answers verified TRUE.Precision reflects the reliability of retained code-integrated solutions; Recall reflects filtering efficiency.Source: D Analysis of Code-integrated… · p. 11
  • Precision is significantly higher than Accuracy across all datasets, which the authors say underscores the effectiveness and generalization of code-integrated verification.Source: D Analysis of Code-integrated… · p. 11

E Finetuning Details

  • Fine-tuning used AdamW, batch size 64, a cosine scheduler, and a 2e−5 learning rate with 50 warm-up steps.The 70B and 34B models used 32 NVIDIA A800 80GB GPUs; Mistral-8x7B used 16, and the 7B, 13B, and 20B models used 8.Source: E Finetuning Details · p. 11

F Analysis of Verification Rounds in Verified Inference

In verified inference, the model verifies solutions and re-solves those judged incorrect, reducing the number of questions needing solutions each round. Accuracy gains become small after two rounds, while further verification can add cost without improving accuracy.

F Analysis of Verification Rounds in Verified Inference

G Examples of Code-Integrated Verification Rationales

The examples show how code-integrated rationales verify solutions by checking conditions or comparing answers from an alternative solution method.

G Examples of Code-Integrated Verification Rationales

  • A correct solution is checked by using its answer to calculate the condition and comparing it with the actual condition.The incorrect example instead solves the question by an alternative method and compares the answers.Source: G Examples of Code-Integrated… · p. 12, Table 13 · p. 15, Table 14 · p. 16
  • Table 11 defines accuracy as the fraction of solutions that are correct, precision as correctness among solutions verified correct, and recall as verified-correct solutions among actually correct solutions.Source: Table 11 · p. 13
  • Table 12 reports MathGenieLM-Llama-2-70B results using varying inference verification-round limits, with k denoting the maximum number of rounds.Source: Table 12 · p. 13
  • Figure 4 illustrates three iterations of Direct Question Augmentation and Iterative Solution Augmentation and Question Back-translation.Source: Figure 4 · p. 14
Loading 2402.16352v2…