PaperlayerView exact revision on arXiv

Source-linked AI summary

ReasonXL: Shifting LLM Reasoning Language Without Sacrificing Performance

Daniil Gurgurov, Tom Röhr, Sebastian von Rohrscheidt, Josef van Genabith, Alexander Löser, Simon Ostermann

arXiv:2604.12378v1cs.CL

TL;DR

LLMs often produce English reasoning even for non-English prompts, while large-scale non-English reasoning data and direct target-language optimization are limited. This paper introduces ReasonXL and adapts a model with supervised fine-tuning followed by reinforcement learning. The adapted models reason in the target language while matching or exceeding the base model on average, and analysis identifies early layers as a causal language-routing bottleneck.

  • Problem

    LLMs often default to English for intermediate reasoning even under non-English prompts, while non-English reasoning traces and direct target-language optimization remain limited.

    Source: §1 · p. 1

  • Method

    The paper introduces a large-scale multilingual reasoning corpus and uses it to adapt SmolLM3-3B through per-language supervised fine-tuning followed by reinforcement learning.

    Source: §3.2 · p. 4, §6 · p. 9

  • Results

    Across four target languages, final models preserve target-language reasoning and match or outperform the base model on average; patching layers 6–8 shifts output toward French (fraction ≈0.8 at the peak).

    Source: §4.1 · p. 6, §5.2 · p. 8

  • Takeaways & Limitations

    Early layers around 6–8 act as a language-routing bottleneck, while deeper layers accumulate the largest weight changes and likely refine reasoning and output quality.

    Source: §5.2 · p. 8

    The adaptation study uses one base model, SmolLM3-3B, and a fixed two-stage pipeline without exhaustive language-specific tuning of hyperparameters, reward weights, or data filtering.

    Source: §6 · p. 9

Abstract

from arXiv · show

Despite advances in multilingual capabilities, most large language models (LLMs) remain English-centric in their training and, crucially, in their production of reasoning traces. Even when tasked with non-English problems, these models predominantly reason in English, creating a fundamental mismatch for non-English usage scenarios. We address this disparity directly with three contributions. (i) We introduce ReasonXL, the first large-scale parallel corpus of cross-domain reasoning traces spanning five European languages (English, German, French, Italian, and Spanish), with over two million aligned samples per language, each comprising prompts, reasoning traces, and final outputs, enabling direct supervision of language-specific reasoning. (ii) Using ReasonXL, we demonstrate that LLMs can be adapted to reason entirely in a desired target language, using a simple two-stage pipeline of supervised fine-tuning (SFT) followed by reinforcement learning with verifiable rewards (RLVR). The resulting models match or exceed baseline performance, with minimal loss in general knowledge and broadly preserved cross-lingual transfer. (iii) We conduct an extensive representational analysis of the adaptation and find a clear functional division across model depth: early layers contain an activation bottleneck that causally determines language identity, while upper layers concentrate the weight and activation changes driven by adaptation. We further find that RLVR achieves greater behavioral divergence from the base model with smaller parameter updates than SFT, suggesting a more efficient representational rerouting despite much smaller weight updates.

1 Introduction

LLMs often reason in English even when prompted in other languages, while non-English reasoning traces and direct target-language optimization remain limited. The paper introduces ReasonXL, evaluates language-specific post-training, and analyzes how adaptation changes model internals.

1 Introduction

  • Models commonly default to English for intermediate reasoning, and existing work has documented language disparities without directly optimizing reasoning in a target language.The gap matters for users seeking transparent, followable reasoning in their native language.Source: §1 · p. 1
  • ReasonXL provides a large-scale, five-language parallel corpus with over two million reasoning traces per language across multiple domains.Each sample includes rich annotations, and the corpus is described as a living resource under active development.Source: §1 · p. 2
  • The paper evaluates language-specific reasoning adaptation through fine-tuning followed by reinforcement learning, including target-language reasoning, knowledge retention, and cross-lingual transfer.The introduction identifies evidence that language-specific post-training can match English-level performance as a missing piece.Source: §1 · pp. 1–2
  • The authors analyze how adaptation reshapes model internals through weight changes and activation patterns across layers.This analysis is presented as enabled by the controlled setup.Source: §1 · p. 2

2 Related Work

Prior work documents English-default reasoning and explores translation, prompting, post-training, and test-time scaling, but large-scale non-English reasoning data remain scarce. This paper pairs a multilingual corpus with post-training and examines adaptation in model representations.

2 Related Work

  • Models often default to English for intermediate reasoning, and reported accuracy is higher in English than in other languages.In-language reasoning is linked to trust, direct inspection of traces, and preservation of linguistic and cultural nuance.Source: §2 · p. 2
  • Prior approaches include translation-based, prompt-based, post-training, and test-time-scaling methods, with reported trade-offs in compliance, accuracy, and consistency.Progress across these approaches is constrained by scarce large-scale non-English reasoning data.Source: §2 · p. 2
  • The paper studies language-specific post-training without an English pivot and examines how adaptation reshapes language representations.Prior interpretability work has largely examined pretrained models in isolation, leaving post-training adaptation an open question.Source: §2 · pp. 2–3

3 Methodology

The methodology builds a curated, translated multilingual reasoning corpus, adapts a base model independently for each target language through supervised fine-tuning and reinforcement learning, and evaluates target-language, out-of-domain, and English knowledge performance.

3.1 Dataset Creation

  • ReasonXL is built from ten existing resources, with English samples annotated and filtered for safety, completeness, reasoning indicators, quality, and domain balance.Qwen3-32B independently translates user input, reasoning traces, and final outputs; manual inspection reports high fidelity in technical terminology, notation, and reasoning structure.Source: §3.1 · pp. 3–4

3.2 Per-Language Reasoning Adaptation Experiments

  • The experiments adapt SmolLM3-3B independently for German, Italian, French, and Spanish, using SFT on each language’s reasoning traces followed by RL with verifiable answers.SFT trains on assistant reasoning and outputs; RL starts from each language-specific SFT checkpoint and uses prompts translated from MATH and GSM8K.Source: §3.2 · p. 4

3.3 Evaluation Datasets

  • Evaluation covers target-language science and non-science tasks plus English general-knowledge benchmarks, with multiple seeded runs and language-specific prompts for final adapted models.The English-subset table is described as showing consistent average science-benchmark gains after SFT, but the supplied passage gives no values.Source: §3.3 · p. 5

4 Reasoning Language Adaptation Results

The two-stage adaptation shifts reasoning into each target language while recovering performance after SFT, with limited general-knowledge loss. Cross-lingual effects vary by adaptation language, but comprehension is broadly preserved.

4.1 Single-Language Adaptation

  • SFT on the English split yields average gains across science benchmarks, validating that the corpus improves strong baselines even in English.Source: §4.1 · p. 5
  • Across all four target languages, GRPO preserves target-language compliance and brings average performance to match or exceed the base model after SFT’s 5–10-point average science-performance drop.The gains also extend to out-of-domain benchmarks Belebele, GMMLU, and Include.Source: Table 3 · p. 6, §4.1 · p. 6

4.2 General Knowledge Shift

  • After SFT+RL, English general-knowledge performance drops by 0.6 points for Italian adaptation to 3.2 points for Spanish, while most individual benchmarks decline by less than 2 points.Source: §4.2 · p. 6, Table 4 · p. 7

4.3 Cross-lingual Performance

  • Transfer varies by adaptation language: Italian and German yield broadly positive transfer, French more consistent degradation, and Chinese improves across all adapted models, while English declines in every setting.Source: §4.3 · p. 6, Table 5 · p. 7
  • The results suggest that changing the surface reasoning language does not substantially disrupt underlying language-agnostic representations.Source: §4.3 · p. 7

5 A Representational Analysis on the Effects of Reasoning Tuning

The analysis compares base, SFT, and final RL checkpoints, primarily on 250 French MGSM prompts. It finds large weight changes in upper layers but a language-switch bottleneck at layers 6–8.

5.1 Weight Change Analysis

  • Weight updates grow toward upper layers, where SFT and RL both shift substantially from Base; RL-over-SFT updates are an order of magnitude smaller and more uniform across depth.The profiles average relative Frobenius-norm changes across attention and MLP component tensors within each layer.Source: §5.1 · p. 7, Figure 1 · p. 8

5.2 Activation Change Analysis

  • Cosine similarity to Base forms a sharp valley at layers 6–8 for both fine-tuned models, while SFT and RL representations remain highly similar throughout most layers.The early dip indicates localized restructuring shared by the fine-tuned checkpoints.Source: §5.2 · p. 8, Figure 2 · p. 9
  • The findings suggest a division of labor: early layers route generation into the target language, while upper layers likely refine reasoning and output quality within that language.Source: §5.2 · p. 8
  • Patching RL hidden states into Base at layers 6–8 shifts French output to ≈0.8, while patching other layers yields ≈0.01, matching clean Base.Source: §5.2 · p. 8, Figure 2 · p. 9

6 Discussion and Conclusion

The discussion presents ReasonXL as an open multilingual reasoning resource and reports that target-language reasoning adaptation can preserve task performance. It also highlights layer-localized adaptation changes and an early causal language bottleneck, while characterizing the experiments as an initial demonstration.

6 Discussion and Conclusion

  • ReasonXL is a public corpus with over 2M aligned samples across five languages, enriched with metadata and intended to support multilingual-reasoning research.Proposed uses include transfer studies, reasoning-trace analysis, curriculum design, and evaluation of language-specific chain-of-thought faithfulness.Source: §6 · p. 9
  • The adaptation study is an initial feasibility demonstration using one base model and a fixed pipeline, without exhaustive language-specific tuning.The authors identify larger models, improved RL strategies, and mixed-language curricula as directions for further work.Source: §6 · p. 9
  • Adaptation changes concentrate in upper layers, while activation patching identifies a causal language-output bottleneck at layers 6–8; RL produces greater predictive divergence despite smaller parameter updates.The three model variants—base, SFT, and RL—also provide a setting for finer-grained circuit- and neuron-level analyses.Source: §6 · p. 9
  • The current ReasonXL release contains approximately 2M samples per language, and the paper reports shifting reasoning to a target language without degrading task performance.The translation pipeline is expanding toward 20–30B tokens per language.Source: §6 · p. 10

A Limitations

The limitations section bounds the adaptation evidence by model, language, translation-validation, and reward-design choices. The translation-prompt figure identifies three fields: input, thinking/reasoning, and output.

A Limitations

  • Experiments cover one 3B model and five typologically related European languages; translation quality lacked human validation, and the RL stage uses a simple composite reward.Generalization beyond the tested scale and languages remains unverified, and translationese artifacts could influence training dynamics.Source: A Limitations · p. 16

B Translation Prompt

  • Figure 3’s translation prompt template has fields for input, thinking/reasoning, and output.The appendix does not specify further field-level distinctions in this caption.Source: Figure 3 · p. 16

C Dataset Samples

The dataset appendix shows example translations and outlines mandatory filtering, domain-dependent quality thresholds, and class-aware downsampling. It also points to tables containing sampling ratios and the translation examples.

C Dataset Samples

  • Table 6 presents example translations from the parallel M-REASON corpus.Source: Table 6 · p. 17

D.1 Mandatory Safety and Integrity Filters

  • Mandatory filters retain safe documents without personal information and require complete content.The appendix also states that non-informational or transactional categories are removed.Source: §D.1 · p. 17
  • Documents must also have reasoning indicators and no commercial-bias label.Source: §D.1 · p. 17

D.2 Domain-Dependent Quality Filtering

  • Strict technical domains require evergreen content, dense information, moderate-to-high educational value, and excellent content quality.Source: §D.2 · p. 18
  • Documents outside strict technical domains receive a relaxed content-quality threshold.The appendix also describes class-aware downsampling and says documents in classes absent from Table 7 are retained without downsampling.Source: §D.2 · p. 18, §D.3 · p. 18

D.3 Class-Balanced Sampling

D.4 Final Dataset Construction

The final dataset is weighted toward dense, educational material, with its largest content groups being non-technical, mathematical, and scientific. The appendix also identifies translation and SFT configuration tables.

D.4 Final Dataset Construction

  • 61.5% of documents are dense and 35.2% adequate; 53.6% have high educational value and 33.1% moderate.Non-technical, mathematical, and scientific content account for 24.8%, 23.6%, and 21.1%, respectively; basic technical material is 17.6%, code 9.4%, data 1.8%, and engineering 1.6%.Source: §D.4 · p. 19

E Translation Hyperparameters

F.1 SFT Hyperparameters

  • The same SFT configuration is used for all four target languages; only the dataset split differs.Source: §F.1 · p. 19

F.2 GRPO Training Details

The appendix specifies Dr. GRPO training and composite reward components, then documents evaluation settings and benchmark-specific answer extraction for science and non-science tasks.

F.2 GRPO Training Details

  • Dr. GRPO removes the KL penalty and uses normalized advantages; the composite reward is a weighted sum of components.Source: §F.2 · p. 19
  • The reward combines symbolic answer correctness, language fidelity, structured formatting, and penalties for repetition.The language score weights reasoning traces more heavily than outputs; repetition penalties target n-gram loops, token flooding, and repeated characters.Source: §F.2 · pp. 19–20
  • Spanish adds a naturalness penalty for atypical inverted-question-mark patterns, while Spanish and French use repetition penalties to counter preliminary length-hacking.Traces shorter than 30 words receive a neutral Spanish naturalness reward.Source: §F.2 · p. 21, Table 11 · p. 21

G Evaluation Details

  • Evaluations use vLLM with bfloat16, a 16,384-token context and output limit, temperature 0.6, and top-p 0.95; runs number five for science and non-science tasks and three for general knowledge.Final adapted models receive the target-language system prompt used in RL; base and SFT models do not.Source: G Evaluation Details · p. 21

G.1 Science Tasks

  • Science-task answer extraction varies by benchmark: MGSM uses fallback numeric parsing, MT-Math100 uses nested-brace-aware boxed parsing, and multilingual GPQA Diamond matches extracted A–D labels.Source: §G.1 · p. 21

G.2 Non-Science Tasks

  • Global MMLU Lite, Belebele, and Include use localized multiple-choice prompts and boxed-answer extraction, with benchmark-specific procedures for deriving gold labels.Source: §G.2 · p. 22

G.3 General Knowledge Tasks

General-knowledge benchmarks are evaluated in English, while Italian, Spanish, and German analyses examine layerwise weight changes and representation behavior during adaptation.

G.3 General Knowledge Tasks

  • General-knowledge benchmarks are evaluated in English to assess knowledge drift, with three runs and a shared inference configuration.The benchmark set includes multiple-choice tasks and BoolQ, with boxed-answer extraction and task-specific fallbacks.Source: §G.3 · p. 22, Table 12 · p. 22

H.1 Italian

  • Italian adaptation produces larger attention and MLP weight updates in upper layers; Figure 5 compares layerwise model similarity and activation-patching effects on target-language output.Source: Figure 4 · p. 23, Figure 5 · p. 23

H.2 Spanish

  • Spanish adaptation produces larger attention and MLP weight updates in upper layers; Figure 7 compares layerwise model similarity and activation-patching effects on target-language output.Source: Figure 6 · p. 24, Figure 7 · p. 24

H.3 German

  • German adaptation produces larger attention and MLP weight updates in upper layers; Figure 9 compares layerwise model similarity and activation-patching effects on target-language output.Source: Figure 8 · p. 25, Figure 9 · p. 25

Related papers

Loading 2604.12378v1…