PaperlayerView exact revision on arXiv

Source-linked AI summary

Parallel Scaling Law: Unveiling Reasoning Generalization through A Cross-Linguistic Perspective

Wen Yang, Junhong Wu, Chong Li, Chengqing Zong, Jiajun Zhang

arXiv:2510.02272v1cs.CLcs.AI

TL;DR

The paper asks whether reasoning learned through English training transfers to other languages, a question left open by prior focus on generalization across tasks or modalities. It evaluates English-centric models and studies parallel-language training, finding distinct transfer patterns and a power-law scaling relationship alongside a gap in monolingual generalization.

  • Problem

    Whether RL-based reasoning learned through English training transfers to other languages remains unclear, despite existing work on generalization across tasks or modalities.

    Source: §1 · pp. 1–2

  • Method

    The authors introduce MTI, evaluate English-centric LRMs across multilingual benchmarks, and study parallel training with parallel data from one to seven languages.

    Source: §1 · p. 2

  • Results

    Parallel training produces a First-Parallel Leap, a power-law Parallel Scaling Law, and a Monolingual Generalization Gap between predicted and actual monolingual performance.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings identify limitations in current LRMs and offer insights for developing more language-agnostic reasoning models.

    Source: §6 · p. 10

    The study focuses on mathematical reasoning, so whether its findings generalize to domains such as coding or agent planning remains unverified.

    Source: A LIMITATIONS AND FUTURE WORK · p. 15

Abstract

from arXiv · show

Recent advancements in Reinforcement Post-Training (RPT) have significantly enhanced the capabilities of Large Reasoning Models (LRMs), sparking increased interest in the generalization of RL-based reasoning. While existing work has primarily focused on investigating its generalization across tasks or modalities, this study proposes a novel cross-linguistic perspective to investigate reasoning generalization. This raises a crucial question: $\textit{Does the reasoning capability achieved from English RPT effectively transfer to other languages?}$ We address this by systematically evaluating English-centric LRMs on multilingual reasoning benchmarks and introducing a metric to quantify cross-lingual transferability. Our findings reveal that cross-lingual transferability varies significantly across initial model, target language, and training paradigm. Through interventional studies, we find that models with stronger initial English capabilities tend to over-rely on English-specific patterns, leading to diminished cross-lingual generalization. To address this, we conduct a thorough parallel training study. Experimental results yield three key findings: $\textbf{First-Parallel Leap}$, a substantial leap in performance when transitioning from monolingual to just a single parallel language, and a predictable $\textbf{Parallel Scaling Law}$, revealing that cross-lingual reasoning transfer follows a power-law with the number of training parallel languages. Moreover, we identify the discrepancy between actual monolingual performance and the power-law prediction as $\textbf{Monolingual Generalization Gap}$, indicating that English-centric LRMs fail to fully generalize across languages. Our study challenges the assumption that LRM reasoning mirrors human cognition, providing critical insights for the development of more language-agnostic LRMs.

1 INTRODUCTION

Whether reasoning learned through English training transfers to other languages remains unclear, despite growing study of reasoning generalization across tasks and modalities. This work measures cross-lingual transfer and examines its determinants, then studies parallel-language training as a way to improve it.

1 INTRODUCTION

  • The paper asks whether reasoning abilities learned through English training generalize to other languages, where language-specific training patterns may matter.The question is motivated by the contrast with human reasoning, which the authors describe as largely independent of language.Source: §1 · pp. 1–2
  • A three-stage study introduces MTI and evaluates 13 English-centric LRMs across 11 languages and four multilingual reasoning benchmarks.The observational study finds transferability varies with the initial model, target language, and training paradigm.Source: §1 · p. 2
  • Interventional studies find that stronger initial English capabilities are associated with greater reliance on English-specific patterns, diminishing cross-lingual generalization.The interventions control for differences in training data, hyperparameters, initial models, and training paradigms.Source: §1 · p. 2
  • Parallel training yields a First-Parallel Leap, a power-law Parallel Scaling Law, and a Monolingual Generalization Gap between predicted and actual monolingual performance.The authors report parallel data from one to seven languages and interpret the gap as evidence that English-centric reasoning does not fully generalize.Source: §1 · p. 2

2 OBSERVATIONAL STUDY

The study finds that cross-lingual reasoning transfer varies across models, training paradigms, and languages, with especially different SFT and RL outcomes in low-resource languages. Interventional and parallel-training studies further identify factors associated with transfer and patterns in how it scales.

2 OBSERVATIONAL STUDY

  • The observational study evaluates open-source reasoning models on diverse multilingual benchmarks to provide a systematic view of cross-lingual reasoning generalization.The selected models include systems fine-tuned with SFT or RPT.Source: §2 · p. 2, §2.1 · p. 2

Interventional Study

  • Interventions associate stronger English-specific reliance with weaker cross-lingual generalization; retained pretrained knowledge and weaker initial models can favor transfer, with model-size patterns varying by evaluation domain.Instruction-tuned models show weaker cross-lingual generalization; 1.5B models improve more on in-domain and other-domain reasoning, while 7B models transfer more robustly across challenging math benchmarks.Source: Interventional Study · p. 3

Parallel Training Study

  • The First-Parallel Leap marks a disproportionate gain from monolingual to bilingual training; further gains follow a diminishing-returns power law, while English-only performance falls short of its prediction.The authors call this discrepancy the Monolingual Generalization Gap and report scaling exponents of β = 0.29 for transferability and β = 0.02 for absolute accuracy.Source: Parallel Training Study · p. 3

2.1 OBSERVATIONAL SETUP

  • The evaluation uses multilingual MATH500, AIME2024, AIME2025, and GPQA-Diamond questions translated from English into ten additional languages.Models are prompted to reason in the user's language; performance measures include accuracy and off-target response rate.Source: Parallel Training Study · pp. 3–4

Observational Study

  • Initial-model properties influence transfer, and the observational findings report that RL yields substantial improvements where SFT degrades performance in low-resource languages.The same training data, paradigms, and hyperparameters can still yield different transfer abilities across initial models.Source: Observational Study · p. 3, §2.2 · p. 4

2.2 RESULTS

  • RL-tuned models generally achieve higher MTI than SFT-tuned models, while low-resource languages show negative transfer with SFT and positive transfer with RL.DeepSeek-R1-Distill-Qwen-7B is the stated exception to the RL-over-SFT MTI pattern; for Swahili and Telugu, RL transfer exceeds that of medium- and high-resource languages.Source: §2.2 · pp. 4–5, Figure 1 · p. 5

3 INTERVENTIONAL STUDY

The interventional studies control experimental factors to examine how training setup and initial model properties shape cross-lingual reasoning transfer. Results differ by initial model type, model family, and model size across multilingual benchmarks.

3 INTERVENTIONAL STUDY

  • Interventional studies control datasets, initial models, and training paradigms to isolate their impacts; observational comparisons cannot isolate causes because model configurations vary.Source: §3 · p. 5, §3.2 · p. 6

3.1 INTERVENTIONAL SETUP

  • The control studies use a curated dataset of 1,000 samples selected from the MATH training set.Source: §3.1 · pp. 5–6
  • GRPO (Shao et al., 2024), a simplified PPO-based RPT algorithm without a value model, uses grouped rollouts and a weighted reward for accuracy, format, and language consistency to guide policy updates.Source: §3.1 · p. 6

3.2 CONTROLLED SETTING AND RESULTS

  • After English training, instruction models achieved the highest reasoning accuracy (Avg: 23.51) and consistency (Off-tag: 0.94), while base and math-specific models had higher MTI (1.95 and 2.12, respectively).Source: §3.2 · p. 6, Table 1 · p. 7
  • On MATH500, English GRPO improved trained- and other-language reasoning for both model families, with cross-lingual transfer inversely correlated with initial capability.Source: §3.2 · p. 6, Figure 2 · p. 7
  • On multilingual GPQA-Diamond, 1.5B models gained significantly across languages, while 7B models showed marginal gains and English performance degradation.Source: §3.2 · p. 7
  • On in-domain MATH500, 1.5B models gained more than 7B models; on challenging AIME24/25, models with stronger initial capabilities transferred more robustly.Source: Figure 3 · p. 7, §3.2 · p. 7

4 PARALLEL TRAINING STUDY

The parallel-training study tests whether adding aligned multilingual problem sets improves reasoning transfer, finding a large initial gain and diminishing returns as parallel languages increase. It also identifies a gap between monolingual performance and the parallel-training scaling pattern.

4 PARALLEL TRAINING STUDY

  • The study proposes “Just Go Parallel”: simultaneous training on bilingual or larger parallel problem sets in different languages.It uses Qwen2.5-7B-Instruct with GRPO-based RPT and aligned samples in English plus seven typologically diverse languages.Source: §4 · p. 8, §4.1 · p. 8

4.1 EXPERIMENTAL SETUP AND RESULTS

  • Models are evaluated on accuracy and MTI across eleven languages using multilingual MATH500, with the number of parallel training languages varied from one to seven.Figure 4 plots performance metrics against the number of training languages, defined as English plus the specified parallel languages.Source: §4.1 · p. 8, Figure 4 · p. 8
  • MTI rises from 1.16 to 2.50 (+1.34), while accuracy rises from 54.24 to 57.87 (+3.63) when moving from monolingual to one parallel language.The gains from one to seven parallel languages are smaller: MTI increases from 2.50 to 3.63 (+1.13), and accuracy from 57.87 to 59.52 (+1.65).Source: §4.1 · p. 8
  • β = 0.29 for transferability versus β = 0.02 for accuracy, suggesting parallel training benefits reasoning transfer more than absolute performance.The fitted power-law curves in Figure 4 model both metrics as a function of parallel-language count, with diminishing returns.Source: Figure 4 · p. 8, §4.1 · p. 9
  • Expected monolingual MTI is approximately 2.00 versus actual 1.16; predicted accuracy is 56.98% versus actual 54.24%, the Monolingual Generalization Gap.The authors interpret the discrepancy as evidence that English-centric monolingual training does not follow the multilingual scaling pattern and may rely on language-specific patterns.Source: §4.1 · p. 9

4.2 ANALYSIS AND DISCUSSION

  • Parallel data supplies an explicit semantic-equivalence signal across languages, unlike unparallel data that only broadens language exposure.Figure 5 compares accuracy differences for parallel and unparallel data training.Source: §4.2 · p. 9, Figure 5 · p. 9
  • Training with Russian yields MTI 2.84, ahead of Bengali at 2.73, German at 2.56, and Chinese at 2.50, with the lowest off-target rate.The authors report only minor variation by selected parallel language and say adding one language consistently improves transferability and multilingual reasoning performance.Source: §4.2 · p. 9, Figure 6 · p. 10

5 RELATED WORK

Related work covers reinforcement learning for reasoning, generalization across tasks and modalities, and cross-lingual transfer. This study distinguishes itself by systematically examining cross-lingual generalization in RL-based reasoning models.

Related Work

6 CONCLUSION

The paper studies cross-lingual reasoning generalization in English-centric LRMs and reports that stronger English-centric models often overfit language-specific patterns. Its parallel-training experiments identify three phenomena that characterize transfer and inform work toward more language-agnostic LRMs.

6 CONCLUSION

  • Observational and interventional studies find that stronger English-centric models often overfit language-specific patterns, limiting cross-lingual transfer.The parallel-training study identifies the First-Parallel Leap, Parallel Scaling Law, and Monolingual Generalization Gap as phenomena characterizing cross-lingual reasoning.Source: §6 · p. 10

REPRODUCIBILITY STATEMENT

The reproducibility statement describes evaluation settings and notes that minor inference variations may remain across hardware or framework versions.

REPRODUCIBILITY STATEMENT

  • Evaluations average 16 sampled generations per question on data-scarce benchmarks, using temperature 0.6, top-p 0.95, and a fixed random seed.The authors note that hardware or inference-framework version differences may still cause minor inference-result variations; code and model weights are slated for release after review.Source: REPRODUCIBILITY STATEMENT · p. 11

A LIMITATIONS AND FUTURE WORK

The work is limited to mathematical reasoning, and the authors say whether its findings generalize to other domains remains to be verified. The causal mechanisms underlying observed transfer patterns are still hypothetical.

A LIMITATIONS AND FUTURE WORK

  • The study focuses on mathematical reasoning; whether its findings extend to coding or agent planning remains unverified.The authors identify applying the findings to low-resource languages as a critical next step.Source: A LIMITATIONS AND FUTURE WORK · p. 15
  • The causal mechanisms behind links between initial capabilities, training strategies, and cross-lingual transfer remain hypothetical.The reasons for RL’s advantage in low-resource languages and the linguistic patterns that hinder transfer remain open questions.Source: A LIMITATIONS AND FUTURE WORK · p. 15
  • Mechanistic interpretability is proposed to investigate internal representations and specific failure modes.The authors frame this as future work toward understanding how reasoning and language are coupled in these models.Source: A LIMITATIONS AND FUTURE WORK · p. 15

C EVALUATION DETAILS AND SETUP

The evaluation uses multilingual versions of four reasoning benchmarks and surveys open-source models spanning several families and sizes.

C.1 MULTILINGUAL REASONING BENCHMARKS

  • Evaluation uses translated MATH500, AIME24/25, and GPQA-Diamond questions across eleven languages.The additional languages are Spanish, Russian, German, French, Bengali, Swahili, Thai, Japanese, Chinese, and Telugu.Source: §C.1 · p. 15
  • MATH500 contains 500 multi-step problems; AIME24/25 are competition datasets, while GPQA-Diamond has 198 multiple-choice science questions.MATH500 spans five mathematical domains, and GPQA-Diamond covers biology, chemistry, and physics.Source: §C.1 · p. 15

C.2 AN OVERVIEW OF OPEN-SOURCE LRMS

  • The evaluated open-source models include DeepSeek-R1-Distill-Qwen, OpenThinker, Simple-RL-Zoo, s1, and DAPO-Qwen, ranging from 1.5B to 32B parameters.Source: §C.2 · pp. 15–16, Table 2 · p. 16

D.1 TRAINING DATASET

The training data includes English questions and a separate Russian set for unparalleled training, while the setup specifies compute, training, and inference choices. Template comparisons led the authors to use the Qwen-Instruct Template as the evaluation default.

D.1 TRAINING DATASET

  • The 1,000 Russian unparalleled-training questions form a separate, non-overlapping set whose type and difficulty distributions closely match the 1,000 English questions.The English questions were sampled from MATH; the Russian set was used to check unparalleled training.Source: §D.1 · p. 16, Figure 7 · p. 16

D.2 EXPERIMENTS ENVIRONMENTS

  • Training and inference used Ubuntu 22.04 with 8 NVIDIA A800 GPUs, VeRL v0.2 for RL tuning, and vLLM 0.8.5 for inference.Source: §D.2 · pp. 16–17

D.3 HYPERPARAMETERS

  • RL training used 4,096-token generations, 1,024-token prompts, a 1 × 10−6 learning rate, batch size 128, and 16 rollouts per question.Inference used temperature 0.6 and top-p 0.95 for 1.5B–14B models; AIME accuracy averaged 16 generations per question, while MATH500 and GPQA used one.Source: §D.3 · p. 17

E.1.1 HOW TO SELECT A TEMPLATE FOR BASE MODEL?

  • On multilingual MATH500, the Qwen-Instruct Template improved reasoning accuracy and language consistency for both base models and became the evaluation default.Source: §E.1.1 · p. 17

E.1.2 THE PERFORMANCE OF INITIAL MODELS

The section compares initial models, tuning paradigms, model families and sizes, and parallel-language settings using multilingual accuracy, off-target rates, and transfer metrics.

E.1.2 THE PERFORMANCE OF INITIAL MODELS

  • Qwen2.5-7B-Instruct leads its 7B base variants in multilingual accuracy and has the lowest off-target rate, while larger Qwen2.5 base models improve with size.Qwen2.5-7B-Instruct also exceeds Qwen2.5-32B in multilingual accuracy; the authors identify instruction-tuned models as advantageous RL starting points.Source: §E.1.2 · p. 18

E.1.3 THE PERFORMANCE OF OPEN-SOURCE MODELS

  • RL-tuned models improve across languages, especially Bengali, Swahili, and Telugu, while most 7B SFT-tuned models degrade on those low-resource languages.Scaling SFT models to 32B brings modest gains, but low-resource degradation persists; DeepSeek-R1-Distill-Qwen-7B is the stated 7B exception.Source: §E.1.3 · p. 18, Figure 8 · p. 19

E.2.1 THE IMPACT OF DIFFERENT MODEL FAMILIES

  • RL improves reasoning across languages for both Qwen2.5 and Llama3.1, with Llama3.1 showing larger gains than Qwen2.5.The authors suggest stronger initial English reasoning may coincide with greater reliance on English-specific patterns and weaker cross-lingual transfer.Source: §E.2.1 · p. 20

E.2.2 THE IMPACT OF MODEL SIZE

  • The 1.5B model has larger relative gains on MATH500 and GPQA-Diamond, whereas the 7B model transfers better to AIME24 and AIME25.The authors describe a trade-off: stronger initial English performance leaves less potential for large relative cross-lingual gains.Source: §E.2.2 · p. 20

E.3.1 THE LANGUAGE SETTINGS IN PARALLEL SCALING LAW

  • The parallel-language settings systematically increase the number of training languages from one to seven.Source: §E.3.1 · p. 20

E.3.2 THE DETAILED RESULTS IN PARALLEL SCALING LAW

  • Tables 9–10 report accuracy and multilingual transfer metrics across languages as the number of parallel training languages varies.Source: §E.3.2 · p. 22

E.3.3 SCALING LAW INTERPRETATION: THE DRIVERS BEHIND THE EXPONENTS

The authors interpret parallel-language scaling as diminishing returns: accuracy gains are small, while transferability grows more strongly. Selected-language results show the largest relative gains for low-resource languages, with outcomes largely robust to language choice.

E.3.3 SCALING LAW INTERPRETATION: THE DRIVERS BEHIND THE EXPONENTS

  • β = 0.02 for accuracy and β = 0.29 for transferability: added languages yield marginal accuracy gains but strengthen cross-lingual abstraction more substantially.The authors attribute the low accuracy exponent to reasoning-task difficulty and interpret the transferability exponent as a “learning-to-learn” capacity whose marginal gains diminish as representations stabilize.Source: §E.3.3 · pp. 22–24
  • The proposed intuition is that one or two parallel languages prompt a shift beyond language-specific surface features, while each further language contributes less marginal information.The authors connect this diminishing return to increasingly robust language-agnostic representations and the First-Parallel Leap.Source: §E.3.3 · pp. 22, 24

E.3.4 THE IMPACT OF SELECTED LANGUAGES

  • For GPQA-Diamond, ru has the largest relative gain, which the authors attribute to Qwen2.5-7B-Instruct’s relatively poor ru performance on that benchmark.Source: §E.3.4 · p. 24
  • Low-resource bn, sw, and te show the largest relative gains across selected parallel languages, while ru, de, and zh accuracy stays comparable; bn accuracy improves substantially when bn is included.The overall outcomes are described as largely robust to parallel-language choice, with bn and ru showing language-specific effects.Source: §E.3.4 · p. 24, Figure 10 · p. 25

F.1 MULTILINGUAL REASONING INSTRUCTION

  • The multilingual reasoning instruction asks the model to think in [LANGUAGE], solve a mathematics problem step by step, and end with a boxed answer.Source: §F.1 · p. 26

F.2 PROMPT HACKING TO FORCE RESPONSE LANGUAGE

  • Prompt hacking uses language-specific prefixes asking the model to begin thinking in a requested language; the appendix lists seven of eleven languages.Source: §F.2 · p. 26

F.3 TEMPLATE FOR R1-LIKE REASONING

  • The R1-like reasoning template specifies an internal monologue followed by an answer, with the final answer enclosed in a box.Source: §F.3 · p. 26

Related papers

Loading 2510.02272v1…