PaperlayerView exact revision on arXiv

Source-linked AI summary

Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations

Nuo Chen, Zinan Zheng, Ning Wu, Ming Gong, Dongmei Zhang, Jia Li

arXiv:2310.20246v5cs.CLcs.AI

TL;DR

Multilingual mathematical reasoning remains underexplored, with scarce training data in low-resource languages. The paper constructs multilingual instruction data and trains MathOctopus using different strategies; parallel-training improves in-domain results, while cross-training is more robust on the out-of-domain MSVAMP test set.

  • Problem

    Training powerful multilingual mathematical reasoning models remains underexplored, and low-resource languages have scarce training data.

    Source: §1 · p. 2

  • Method

    The paper translates English GSM8K training data into nine languages, calibrates and verifies the corpora, and uses them to build MGSM8KInstruct and train MathOctopus.

    Source: §1 · p. 2

  • Results

    MathOctopus outperforms other open-source models; parallel-training achieves better in-domain results, while cross-training shows better robustness on out-of-domain MSVAMP.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The paper concludes that multilingual math instruction data can support mathematical reasoning across languages and improve performance in English as well.

    Source: §1 · p. 2, §5 · p. 8

    xRFT is currently applied only to 7B and 13B models because inference is costly.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Existing research predominantly focuses on developing powerful language learning models (LLMs) for mathematical reasoning within monolingual languages, with few explorations in preserving efficacy in a multilingual context. To bridge this gap, this paper pioneers exploring and training powerful Multilingual Math Reasoning (xMR) LLMs. Firstly, by utilizing translation, we construct the first multilingual math reasoning instruction dataset, MGSM8KInstruct, encompassing ten distinct languages, thus addressing the issue of training data scarcity in xMR tasks. Based on the collected dataset, we propose different training strategies to build powerful xMR LLMs, named MathOctopus, notably outperform conventional open-source LLMs and exhibit superiority over ChatGPT in few-shot scenarios. Notably, MathOctopus-13B reaches 47.6% accuracy which exceeds ChatGPT 46.3% on MGSM testset. Beyond remarkable results, we unearth several pivotal observations and insights from extensive experiments: (1) When extending the rejection sampling strategy to the multilingual context, it proves effective for model performances, albeit limited. (2) Employing parallel corpora for math Supervised Fine-Tuning (SFT) across multiple languages not only significantly enhances model performance multilingually but also elevates their monolingual performance. This indicates that crafting multilingual corpora can be regarded as a vital strategy for enhancing model performance in a specific language, especially in mathematical reasoning tasks. For instance, MathOctopus-7B improves its counterparts that trained on English from 42.2% to 50.8% on GSM8K testset. Codes are available at https://github.com/microsoft/MathOctopus.

1 Introduction

The paper addresses the limited exploration of multilingual mathematical reasoning and the scarcity of low-resource training data by translating and validating multilingual math instruction data. It uses that data to train MathOctopus and examines multilingual rejection sampling as an additional strategy.

1 Introduction

  • 42.4% to 50.8%: MathOctopus-7B improves LLaMA2-7B accuracy on English GSM8K after multilingual instruction tuning.The authors attribute the gain to stronger generalization and reasoning knowledge from other languages.Source: §1 · p. 2
  • Multilingual rejection sampling fine-tuning (xRFT) yields limited additional multilingual gains and may weaken generalization as the augmentation data grows.The method augments MGSM8KInstruct with correct reasoning paths generated through multilingual rejection sampling.Source: §1 · p. 2
  • MGSM8KInstruct provides multilingual math instruction data, while MSVAMP adds an out-of-domain multilingual test set for robustness evaluation.The training dataset uses translated, calibrated, and human-verified corpora; MGSM8KInstruct includes parallel-training and cross-training settings.Source: §1 · p. 2
  • MathOctopus models trained with different strategies improve multilingual mathematical reasoning, including in low-resource languages, and also improve English performance.The paper fine-tunes open-source LLaMA-family models on MGSM8KInstruct.Source: §1 · p. 2
  • The paper identifies multilingual supervised fine-tuning as a strategy for enhancing mathematical reasoning proficiency in LLMs.It investigates how data volume and training strategies relate to model performance.Source: §1 · p. 2

2 Methodology

The methodology extends mathematical reasoning to multilingual question-and-answer settings and constructs MGSM8KInstruct from translated GSM8K data. It then uses parallel and cross-training, with multilingual rejection sampling as an augmentation strategy, to train MathOctopus models.

2.1 Problem Formulation

  • The task extends mathematical reasoning to questions and chain-of-thought responses in a target language, including when the question is presented only in English.The formulation represents questions, optional answer choices, and ground-truth answers as D = {Qi, Oi, Ai}.Source: §2.1 · p. 3

2.2 MGSM8KInstruct

  • MGSM8KInstruct contains about 73.6k samples built from GSM8K problems and reasoning responses translated into nine target languages alongside English.The study covers ten languages: Bengali, Chinese, French, German, Japanese, Russian, Spanish, Swahili, Thai, and English.Source: §2.2 · pp. 3–4
  • Native speakers checked 500 samples per language, yielding 91.2% overall agreement; formula checks and discarding cases after five consecutive translation errors further verify translations.The translation prompts preserve names, formulas, and Arabic numerals and provide two examples per language.Source: §2.2 · pp. 3–4

2.3 MathOctopus

  • MathOctopus uses parallel-training with questions and answers in the same language or cross-training with English questions and answers in another language.MathOctopusP and MathOctopusC denote the respective training variants.Source: §2.3 · p. 4, Table 4 · p. 5
  • Multilingual rejection sampling filters incorrect answers and formulas, then adds distinct correct reasoning paths sampled across languages and models to the SFT data.The study samples 25 inferences per language from each of the 7B and 13B models, yielding 50 samples per question in each language.Source: §2.3 · p. 4

3 Experiments

Experiments evaluate MathOctopus on in-domain MGSM and out-of-domain MSVAMP, comparing training strategies and testing formats. MathOctopus outperforms open-source baselines, while results show multilingual SFT can also benefit monolingual performance.

3.1 Evaluation Datasets

  • MGSM has 250 test samples per language, translated from an English GSM8K test-set subset into ten other languages; MSVAMP translates 1,000 SVAMP questions into ten languages, with fidelity reviewed by native speakers.SVAMP is described as out-of-domain source data.Source: §3.1 · p. 5

3.2 Baselines

  • Comparisons include LLaMA base, RFT, MAmmoTH, and WizardMath, alongside ChatGPT and GPT-4 evaluations.ChatGPT is tested zero-shot and with two-shot English chain-of-thought examples.Source: §3.2 · pp. 5–6

3.3 Main Results

  • Existing 7B models achieve about 20%–23% accuracy on MGSM and show a marked high-resource versus low-resource language gap.The paper attributes the gap largely to low-resource languages’ limited representation in foundational corpora; MathOctopus improves results in Thai and Swahili.Source: §3.3 · pp. 6–7
  • At 7B, MathOctopus raises LLaMA’s MGSM accuracy from 22.6% to 41.9%, and MathOctopusP-13B surpasses ChatGPT on MGSM.Parallel-training performs better in-domain, while cross-training shows stronger out-of-domain generalization.Source: §3.3 · p. 7
  • xRFT gains are modest: about 1%–2% for MathOctopusP on MGSM, below 1% on MSVAMP, and potentially negative for MathOctopusC on MGSM.Its contribution is especially limited on out-of-domain tests.Source: §3.3 · p. 7

3.3.1 Training-Testing Consistencies

  • MathOctopusC outperforms MathOctopusP in cross-test settings as model size grows from 7B to 33B, despite their different performance under in-domain testing.MGSM and MSVAMP use parallel-test data, where question and answer are in the same language; matching test and training formats is important to performance.Source: §3.3.1 · p. 7

3.3.2 Multilingual SFT can generally benefit Monolingual SFT

  • On English GSM8K and SVAMP, MathOctopusC outperforms MathOctopusP at 7B, with reported comparisons of 50.8% vs. 49.3% and 49.3% vs. 46.8%.The authors surmise cross-training intensifies English comprehension.Source: §3.3.2 · p. 7
  • On Spanish, Chinese, and low-resource Thai test subsets, MathOctopus surpasses monolingual SFT models trained in their respective languages.The authors conclude that multilingual SFT can improve performance in a model’s native language for mathematical reasoning.Source: Figure 3 · p. 7, §3.3.2 · p. 8

4 Discussion

The discussion examines xRFT data quantity and asks about transferability and alternative training pairings. Its xRFT experiments find generally modest, model- and dataset-dependent effects.

4.1 RQ1: Influence of xRFT Data Volume

  • Larger xRFT corpora generally improve MathOctopusP in most cases, especially on MGSM, but gains are modest and MathOctopusC can worsen on MSVAMP as k increases.The tested sampling counts are k = 10, 30, and 50.Source: §4.1 · p. 8
  • The authors suggest multilingual SFT may already provide diverse reasoning paths through different-language versions of the same solution, limiting xRFT’s additional benefit.They note that supplementary xRFT may lead to overfitting.Source: §4.1 · p. 8

5 Conclusion

The paper introduces multilingual mathematical reasoning models trained on MGSM8KInstruct and reports distinct strengths for parallel- and cross-training. It also finds multilingual rejection sampling has limited effect and that aligned bilingual corpora improve native-language mathematical performance.

5 Conclusion

  • MathOctopus models trained on MGSM8KInstruct outperform other open-source models, with parallel-training stronger in-domain and cross-training more robust on out-of-domain MSVAMP.The dataset contains ten languages; multilingual rejection sampling has a limited effect on xMR tasks.Source: §5 · p. 8
  • Aligned bilingual question-answer corpora significantly improve models’ mathematical capabilities in their native language.Source: §5 · p. 8
  • The paper identifies exploring additional methodologies and diverse parallel corpora, potentially involving RLHF, as future work.Source: §5 · p. 9

Limitations

The paper identifies limits in model scale, the scope of multilingual rejection fine-tuning, and uncertainty about adding languages. Its related-work review situates multilingual mathematical reasoning among work on mathematical reasoning, instruction tuning, and multilingual models.

Limitations

  • MathOctopus has not yet been developed on larger backbones, including LLaMA 2-70B and LLaMA-Coders.Source: Limitations · p. 9
  • xRFT is limited to 7B and 13B models because inference is costly, leaving its efficiency on more performant models for future study.Source: Limitations · p. 9
  • Whether adding more languages to MGSM8KInstruct benefits current models remains unclear.Source: Limitations · p. 9

A Related Works

  • Prior mathematical-reasoning work uses CoT, self-consistency, diverse solution generation, and rejection sampling; Yuan et al. (2023a) apply rejection sampling to augment math fine-tuning data.Source: A Related Works · p. 12
  • Instruction-tuning research extends from general-purpose and synthetic data to multilingual models, but multilingual mathematical reasoning remains less explored.The review cites multilingual instruction tuning for MMLU tasks (Chen et al., 2023d) and interactive translation tasks (Zhang et al., 2023).Source: A Related Works · p. 12

B Experimental Setup

Experiments use LLaMA-2 7B–13B and LLaMA-1 33B backbones, with fixed training and decoding settings across models.

B Experimental Setup

  • The experiments use open-source LLaMA-2 7B–13B and LLaMA-1 33B as MathOctopus backbones.Source: B Experimental Setup · p. 12
  • Training uses a learning rate of 2e-5, 3 epochs, maximum length 512, and batch sizes of 8, 4, and 2 for models from 7B to 33B.Source: B Experimental Setup · p. 12
  • Testing uses a maximum output of 512 tokens and temperature 0.0, with the same prompt for MathOctopus models.Source: B Experimental Setup · p. 12

C CrowdSourcing Verification of MSVAMP

The MSVAMP translations were checked by native speakers, and the authors report high agreement as evidence of reliable translation quality.

C CrowdSourcing Verification of MSVAMP

D.1 RQ2: Targeted Language Training: Limited Broader Linguistic Reasoning Gains

Training on selected languages improves performance in those languages, but effects on languages absent from training vary. The authors suggest grammatical-structure differences may explain this variation.

D.1 RQ2: Targeted Language Training: Limited Broader Linguistic Reasoning Gains

  • Thai accuracy rises from 4% to 36.8% after training on the selected languages.The training languages were Spanish, Chinese, Thai, and Swahili.Source: §D.1 · p. 13
  • For languages unseen in training, outcomes vary: Japanese and French improve, while German and Russian decline.The authors suggest grammatical-structure disparities may predominantly account for the variation (Chen et al., 2023a).Source: §D.1 · p. 13
  • The experiment asks whether training on a subset of languages can enhance mathematical reasoning across all languages.It uses two high-resource languages, Spanish and Chinese, and two low-resource languages, Thai and Swahili.Source: §D.1 · p. 13

D.2 RQ3: Training Variability: Beyond English-centric Questions

The section compares expanded mixed-training strategies with MathOctopusP and describes the translation prompt used in the experiments. Despite larger training volumes, neither mixed strategy surpasses MathOctopusP.

D.2 RQ3: Training Variability: Beyond English-centric Questions

  • MathOctopusM combines cross-training and parallel-training data, while MathOctopusM-All also randomly pairs languages to expand the original training data tenfold.The language pairs are drawn from MGSM8KInstrucT.Source: §D.2 · p. 13
  • Neither mixed strategy surpasses MathOctopusP, despite doubling or expanding training volume tenfold; MathOctopusM-All slightly underperforms MathOctopusM.The authors suggest the expanded data may be unnecessary for learning cross-language alignment and reasoning, and that rudimentary expansion may induce overfitting.Source: Figure 5 · p. 13, §D.2 · p. 13

D.3 Prompts for ChatGPT Translation

  • The ChatGPT translation prompt uses a replaceable lang placeholder, with Spanish translation examples shown.Table 11 presents the translation prompts used in the experiments.Source: §D.3 · p. 13

D.4 Generated Cases by MathOctopus

The appendix provides generated-response examples and experimental prompts, including comparisons between LLaMA and MathOctopus at the 7B level. It also identifies language settings for MathOctopusP examples.

D.4 Generated Cases by MathOctopus

  • The appendix includes training and testing prompts, translation prompts, and 2-shot prompts for ChatGPT and GPT-4.These are presented in Tables 9, 11, and 12, respectively.Source: Table 9 · p. 14, Table 11 · p. 14, Table 12 · p. 14
  • MathOctopusP (En-Sw-Th) examples are associated with training in English, Swahili, and Thai.Table 10 is described as reporting model performance on the MGSM test set and defines an En-Zh-Es training setting.Source: §D.4 · p. 14, Table 10 · p. 14
  • Tables 13 and 14 compare generated LLaMA and MathOctopus responses at the 7B level on MGSM.The passage identifies generated examples in various languages but gives no comparative outcome.Source: §D.4 · p. 13, Table 13 · p. 15, Table 14 · p. 16
Loading 2310.20246v5…