PaperlayerView exact revision on arXiv

Source-linked AI summary

What Does Post-Training Change in Multilingual Reasoning?

Hongyang Li, Xiao Li, Caesar Wu, Grégoire Danoy, Pascal Bouvry

arXiv:2609.37104v1cs.CLcs.AI

TL;DR

A model can solve a problem yet fail to deliver a complete solution in the user’s requested language, and prior evaluations can miss distinct delivery failures. The paper tracks correctness, language adherence, termination, and efficiency across Qwen3 post-training stages, finding that SFT and RL shift the dominant bottleneck and that correctness-gated language reward retains delivery.

  • Problem

    Correct answers with English reasoning or non-terminating target-language traces are not successful delivery, yet final-answer accuracy alone cannot distinguish these failures.

    Source: Introduction · p. 1

  • Method

    The study follows Qwen3 checkpoints through SFT controls and RL reward formulations, jointly evaluating correctness, requested-language reasoning, termination, and delivery efficiency.

    Source: Introduction · p. 1, Setup · p. 2

  • Results

    SFT restores target-language reasoning but brings non-English looping; RL restores termination at equal accuracy, and only correctness-gated language reward retains delivery.

    Source: Introduction · p. 1

  • Takeaways & Limitations

    The reported post-training path uses SFT to repair adherence, RL to restore termination, and correctness-gated reward to improve end-to-end multilingual delivery.

    Source: Introduction · p. 2

    Each run uses one seed and evaluation covers 70 translated problems; the SFT controls and gated endpoint differ in configuration, making comparisons diagnostic rather than fully matched causal ablations.

    Source: Limitations · p. 7

Abstract

from arXiv · show

Open-source reasoning models provide unequal access to reasoning capability across languages. When a model can solve a problem but cannot deliver a complete solution in the user's language, language becomes an access barrier rather than merely a source of performance variation. We audit Qwen3 checkpoints on competition-mathematics tasks in eleven languages. Across the ten non-English languages, only 15.4-17.9% of problems receive a correct, terminating solution with visible reasoning in the requested language in any of 16 samples, compared with 92.9% in English. To identify the source of this disparity, we evaluate thirteen endpoints from one model family, spanning released checkpoints, multilingual supervised fine-tuning (SFT) at two scales, controlled SFT ablations, and three reinforcement-learning (RL) reward formulations. We jointly track correctness, language adherence, termination, and delivery efficiency. The dominant bottleneck shifts across post-training stages. Released models often reason in English. Multilingual SFT restores target-language reasoning, but accuracy declines across multilingual, English-only, and single-language SFT runs, showing that this cost is not specific to multilingual mixing; non-English reasoning traces additionally become prone to non-terminating loops. RL restores termination in both arms at no cost in accuracy, but only the arm whose reward includes a language term delivers: rewarding correctness alone returns the model to English. Together, these stages establish a constructive post-training path from English-pivoted capability to multilingual reasoning that is reliably delivered.

Introduction

Successful multilingual reasoning requires a correct, complete solution in the requested language, but existing evaluations often miss distinct failures in correctness, language adherence, and termination. This study tracks those outcomes across post-training stages within one Qwen3 model family.

Introduction

  • Correct final answers can still be undelivered when visible reasoning is in English or a target-language response fails to terminate.Final-answer accuracy alone does not establish that a user receives a correct, complete solution in the requested language.Source: Introduction · p. 1
  • Prior studies often examine multilingual accuracy, language adherence, efficiency, and reliability separately, leaving unclear which failure mode dominates at each training stage.The paper follows released checkpoints, SFT controls, and RL formulations under a shared evaluation protocol.Source: Introduction · p. 1
  • The study defines delivery as the conjunction of correctness, requested-language reasoning, and termination, and measures efficiency as token cost per joint success.Reporting the components alongside their conjunction is intended to reveal when an intervention repairs one failure mode while worsening another.Source: Introduction · p. 1
  • The analysis separates a general SFT accuracy cost from non-English looping and finds that only correctness-gated language reward holds accuracy while improving end-to-end delivery.The stated post-training path uses SFT to repair adherence, RL to restore termination, and a correctness-gated reward to retain delivery.Source: Introduction · pp. 1–2

Related Work

Prior work identifies several distinct sources of multilingual reasoning gaps, including English-centric traces, accuracy trade-offs, and repetition loops. This paper studies these failure modes together across post-training stages in one model family.

Related Work

  • Prior work reports English-centric internal representations (Wendler et al. 2024; Schut, Gal, and Farquhar 2025) and visible reasoning that defaults to high-resource languages (Tam et al. 2025).Other studies contrast English-pivoted and target-language traces and describe implicit translation at input and output stages.Source: Related Work · p. 2
  • Prior evaluations measure multilingual accuracy, language consistency, reasoning length, or joint success, while prompting for target-language reasoning can reduce accuracy.The passage also notes accuracy reductions from suppressing within-trace language mixing and describes related benchmarks and measures (Hwang et al. 2025; Zhan et al. 2026).Source: Related Work · p. 2
  • Other work reports that limited long-chain supervision can impair small models and subsequent RL, and that reasoning models may enter repetition loops.Multilingual RL studies couple language behavior and task success through consistency objectives, language-adaptive guidance, cross-lingual answer agreement, or reasoning-language targets (Zhang et al. 2026; Fan et al. 2026; Elhady, Agirre, and Artetxe 2026; Gurgurov et al. 2026).Source: Related Work · p. 2
  • Because prior work largely studies these failure modes separately, this paper analyzes one model family across released checkpoints, SFT, and RL under a fixed evaluation.The controlled stage-by-stage comparison identifies which bottleneck dominates at each stage and how interventions change it.Source: Related Work · p. 2

Setup

The study evaluates Qwen3 endpoints under a shared multilingual mathematics protocol, with controlled SFT and RL variants. It defines joint delivery and efficiency measures to compare whether responses are correct, in-language, and complete.

Setup

  • Thirteen Qwen3 endpoints include released checkpoints, multilingual SFT at two scales, English-only and single-language SFT controls, and three RL checkpoints from two arms.The RL arms comprise a correctness-only control and a two-phase target-language procedure with additive and correctness-gated rewards.Source: Setup · p. 2
  • All endpoints are tested on 70 competition-mathematics problems in each evaluation language, with 16 samples per problem and a 24,576-token evaluation budget.The primary evaluation covers English, Chinese, Spanish, French, Arabic, and Russian; joint success is computed per response rather than from marginal rates.Source: Setup · p. 2
  • Multilingual SFT uses 43,218 filtered traces for Qwen3-4B and a separately constructed 56,307-trace corpus for Qwen3-8B; controls include English-only and language-specialist runs.Target-language traces are translated from English mathematics traces and filtered for answer preservation, reasoning structure, language, and repetition.Source: Setup · p. 2
  • Figure 1 traces Qwen3-4B from the released checkpoint through multilingual SFT to correctness-gated RL: SFT restores target-language delivery but introduces non-termination, and RL restores completion while retaining delivery.The rightmost group reports the six-language mean.Source: Figure 1 · p. 3
  • The RL comparison uses a correctness-only reward and two language-reward phases; the additive phase can award language credit to incorrect responses, while the gated phase gives incorrect responses zero reward.The gated phase continues from the additive checkpoint, with a different response cap and batch size.Source: Setup · pp. 2–3
  • Delivery efficiency counts joint successes per 1,000 generated tokens, including attempts that never deliver, and normalizes non-English efficiency against English from the same checkpoint.The setup states that 100% denotes parity with English; it also reports raw-token results and efficiency conditioned on delivered responses.Source: Setup · p. 3

The Delivery Problem at Release

Released Qwen3 checkpoints often solve non-English mathematics problems but fail to deliver the solution in the requested language. The gap persists across five primary and five training-absent languages, while varying by language and checkpoint.

The Delivery Problem at Release

  • At Qwen3-4B, five primary non-English languages score 90.6% C@16 but 35.7% J@16; Qwen3-8B scores 92.6% correctness and 30.6% delivery.Both are close to English correctness of 92.9%, so the missing capability is not primarily mathematical.Source: The Delivery Problem at Release · p. 3
  • Adherence, not termination, is the binding constraint: Spanish, French, and Arabic adherence is at most 0.1% at 4B and zero at 8B, while Chinese adheres almost perfectly.Russian adherence is 99.2% at 4B but 34.6% at 8B, making delivery dependent on the language–checkpoint pair.Source: The Delivery Problem at Release · p. 3
  • Across all ten non-English languages, Qwen3-4B reaches 84.3% C@16 but 17.9% J@16, while Qwen3-8B reaches 89.0% and 15.4%, respectively.The ten include five languages absent from all training corpora.Source: The Delivery Problem at Release · p. 3
  • On Qwen3-8B, Swedish shows a 94.3-percentage-point correctness–delivery gap: 94.3% correct and none delivered.An English problem followed by a request for Chinese, Spanish, French, or Arabic reasoning also yields zero target-language adherence at released Qwen3-4B in a separate probe.Source: The Delivery Problem at Release · p. 4

Supervised Fine-Tuning

Multilingual SFT sharply improves target-language adherence and delivery, but shifts the dominant bottleneck to non-English termination. The accuracy decline also appears in English-only and single-language SFT controls, rather than being specific to multilingual mixing.

Supervised Fine-Tuning

  • Across five primary non-English languages, Qwen3-4B adherence rises from 39.8% to 99.4%, and J@16 from 35.7% to 84.6%; Qwen3-8B J@16 rises from 30.6% to 86.3%.The improvement holds across all five target languages.Source: Supervised Fine-Tuning · p. 4
  • Mean non-English C@16 falls from 90.6% to 85.1% at 4B and from 92.6% to 86.9% at 8B after SFT.Conditional English correctness falls from 86.2% at release to 74.1% after multilingual SFT and 74.5% after English-only SFT; five single-language specialists range from 73.7% to 76.2%.Source: Supervised Fine-Tuning · p. 4
  • At 4B, mean non-English termination failure rises from 4.7% to 21.1%, while English falls from 6.8% to 3.2%; Qwen3-8B repeats the non-English rise from 5.8% to 18.2%.The increase is driven by detected looping across target languages, not token-budget exhaustion; English termination failure again improves at 8B.Source: Supervised Fine-Tuning · pp. 4–5
  • Relative non-English delivery efficiency rises from 47% to 73% of English at 4B and from 27% to 78% at 8B, with remaining cost carried primarily by attempts that never deliver.Conditioned on successful delivery, non-English traces are near parity with English; SFT therefore shifts the dominant bottleneck from adherence to termination.Source: Supervised Fine-Tuning · p. 5

Reinforcement Learning

Both RL arms restore termination and reach similar accuracy, but delivery depends on reward design. The target-language procedure maintains adherence, while the correctness-only control returns the model to English; gating ensures language credit is earned by solving the task.

Reinforcement Learning

  • Both RL arms reduce mean non-English termination failure from 21.1% after SFT to 0.3% and 0.5%, with C@1 of 60.5% and 60.1%, respectively.The comparison starts from the same multilingual SFT checkpoint and contrasts correctness-only reward with a two-phase target-language procedure.Source: Reinforcement Learning · p. 5, Figure 5 · p. 6
  • The correctness-only control reaches 8.7% adherence and 15.4% J@16, while the target-language procedure reaches 99.8% adherence and 81.7% J@16 at equal accuracy.Relative delivery efficiency rises from 73% to 102% of English under the target-language procedure.Source: Reinforcement Learning · p. 5, Figure 5 · p. 6
  • In the additive phase, incorrect trajectories receive 44%–82% of all reward paid even as language quality nears 100% and accuracy remains low.The gated phase withdraws that language credit and continues training, requiring the language-compliant reward to be earned by solving the task.Source: Reinforcement Learning · p. 5

Analysis

Post-training changes which failure limits multilingual delivery: adherence, termination, or language choice. Efficiency reaches parity with English for delivered responses, while an English problem plus a target-language instruction usually does not elicit target-language reasoning.

Analysis

  • Adherence limits released models, termination becomes binding after SFT, and RL repairs termination while correctness gating prevents a trade for language-compliant errors.The dominant bottleneck moves as each stage removes the condition limiting the preceding stage.Source: Analysis · p. 6
  • Adjusted non-English delivery efficiency rises from 73% of English after SFT to 102% at the end of the target-language procedure.Conditioned on delivered responses, raw-token efficiency reaches parity with English; the remaining end-to-end penalty is primarily from attempts that consume tokens without delivering a solution.Source: Analysis · p. 6, Table 3 · p. 7
  • The problem language is a stronger cue than the requested reasoning language; gated RL rarely elicits Chinese, Spanish, French, or Arabic reasoning from English problems.Russian reaches 94.2% after SFT in this probe, but gated RL returns it to 9.2%.Source: Analysis · p. 6, Table 4 · p. 7
  • Multilingual post-training redistributes correlated failure modes rather than uniformly improving a single capability.Evaluating only accuracy, adherence, or termination can miss both the bottleneck removed by an intervention and the failure that replaces it.Source: Analysis · pp. 6–7

Limitations

The conclusions concern visible multilingual delivery in one model family and one task domain; several comparisons are diagnostic rather than fully matched causal ablations.

Limitations

  • The conclusions are limited to visible multilingual delivery in one Qwen3 family on competition mathematics.The released checkpoints had already undergone substantial mathematical post-training, and the interventions target delivery rather than general reasoning capability.Source: Limitations · p. 7
  • Each run uses one seed and evaluation covers 70 translated problems; mismatched control settings make the comparisons diagnostic, not fully matched causal ablations.The English-only and specialist SFT controls differ in data volume and training configuration, while the gated endpoint continues the additive run with a different response cap and batch size.Source: Limitations · p. 7

Supplementary Material

The supplement specifies how the evaluation measures correctness, language adherence, termination, and uncertainty. Its protocol uses localized competition-mathematics problems and problem-clustered intervals.

A Evaluation Protocol and Instrument Checks

  • Primary experiments use 70 competition-mathematics problems, six localized prompts, 16 samples per problem, temperature 0.7, and a 24,576-token response budget.The five-language mean covers Chinese, Spanish, French, Arabic, and Russian; English is the control, and the released-checkpoint audit adds five languages with no fine-tuned endpoints.Source: A Evaluation Protocol and Instrument… · p. 9
  • Correctness is scored from the extracted final answer, while adherence requires at least half of sampled reasoning segments to be classified as the requested language.The adherence measure is cross-checked by a blind LLM judge on 609 non-English traces, with Pearson r = 97.9% against the continuous target-language fraction.Source: A Evaluation Protocol and Instrument… · p. 9
  • Termination requires avoiding both the response cap and repetition above a language-agnostic threshold; pass@k intervals resample problems as blocks.Cap exhaustion and looping are reported separately, and paired comparisons use the same problem-language cells with Holm correction within each five-language family.Source: A Evaluation Protocol and Instrument… · p. 9

A.1 Released-checkpoint results and uncertainty

  • Table S1 reports accuracy and joint delivery with 95% problem-clustered bootstrap intervals, alongside budget exhaustion, looping, and mean tokens over all attempts.The two termination columns can both apply to a trace.Source: Table S1 · p. 9

A.2 Robustness beyond the primary languages

The supplementary material tests whether released-checkpoint findings extend to additional languages, probes language cues, and documents corpus construction. The probe's absolute values are not comparable with endpoint tables.

A.2 Robustness beyond the primary languages

  • Across ten released-checkpoint cells in five additional languages absent from constructed corpora, visible target-language reasoning is essentially absent; the only nonzero J@16 is 1.4% for Swahili on Qwen3-8B.The additional languages span Latin, Tamil, and Arabic scripts, and two independent instruments agree on the absence of target-language reasoning.Source: §A.2 · p. 9, Table S2 · p. 10

A.3 Input-language versus instruction probe

  • The separate forward probe uses English problem text plus an instruction to reason in the target language; its absolute values are not comparable with endpoint tables.Each of seven runs draws its own 30 problems and covers six languages at K = 4, so only within-probe contrasts transfer.Source: §A.3 · p. 10
  • Correctness remains essentially flat across probe conditions (47%–72% after SFT; 71%–86% at release), so refusal to switch language is not a failure to solve.The reported table covers only the Qwen3-4B lineage under the forward condition.Source: §A.3 · p. 10

B.1 Supervised corpus construction

  • The SFT corpus starts from 45K English mathematics traces assigned round robin across six languages; translated traces pass gates for answer preservation, structure, language fidelity, and repetition.The 4B corpus has 43,218 traces, while the separately constructed 8B replication has 56,307.Source: §B.1 · pp. 10–11

B.2 Translator selection

The appendix validates its translation pipeline and training data, documents the training and reproducibility setup, and reports controls and scope checks for the staged results.

B.2 Translator selection

  • Qwen3-14B passed the .98 number-recall mark in all five languages and scored 4.85 vs. 2.91 in blind faithfulness, so it was selected over Hunyuan-MT-7B.The judge comparison recorded 209 wins, 19 losses, and 22 ties; Qwen3-14B's Chinese target-script purity was .877 despite the high faithfulness score.Source: §B.2 · p. 11, Table S4 · p. 11
  • Translated traces retained 99.6–100% of boxed answers and 98.8–100% of their mathematics, while judged coherence was 4.47–4.90 against 4.88 for English sources.Moderate or severe English leakage affected 4.0–6.0% of four languages and 16.0% of Arabic traces; the authors say this cannot explain released-checkpoint results measured before the corpus existed.Source: §B.3 · p. 11, Table S5 · p. 12

B.4 Training recipes

  • SFT minimizes response-token negative log-likelihood; RL uses PPO, with correctness-only and additive runs sharing a configuration and the gated run continuing the additive run from step 200.The multilingual SFT runs used batch 16 on 16 GPUs, versus batch 4 on 4 GPUs for English-only and specialist controls; the gated RL run changed response cap and batch size.Source: §B.4 · p. 12

B.5 Reproducibility statement

  • The artifact retains generated traces and per-sample measurements, while independent scripts produce machine-readable outputs and run logs; the main-paper training ledger is manually transcribed.LLM-judge outputs are cached, and no item is removed from the fixed 70-problem evaluation set.Source: §B.5 · pp. 12–14

C.1 SFT controls and detailed paired results

  • The SFT controls compare multilingual training with English-only and single-language runs, and the paired-delta summary says English-default languages drive delivery gains while correctness declines in English and target languages.The compact mixed–specialist comparison pairs each specialist with the multilingual model on its own language or English.Source: Table S9 · p. 14, Table S8 · p. 14, §C.1 · p. 14

C.2 Is a fluent wrong answer better than a loop?

  • Terminating wrong answers can improve efficiency over loops because loops consume the full 24,576-token budget and return nothing, though confident wrong answers may mislead more readily.Delivered samples stay at 60%, while the share of in-language errors changes from 18% to 39%; joint success does not move because both outcomes are failures under J.Source: §C.2 · p. 15

C.3 End-to-end delivery and state distributions

  • Problem-level correctness correlations across language versions range from 0.819 to 0.988, with a mean of 0.907, indicating that the same problems are hard across languages.The weakest pairs involve Russian, also the language with the largest encoding factor.Source: §C.3 · p. 15, Table S13 · p. 15
  • Fourteen evaluation problems overlap the RL corpus; restricting to the clean 56 lowers J@16 by 2.4 points at the SFT parent and 0.4–3.5 points at RL endpoints.The largest excess over the parent is 1.1 points, and the three RL arms share the overlapping items, so overlap cannot account for differences between them.Source: §C.3 · p. 15, Table S14 · p. 16

C.4 Efficiency

The efficiency analysis separates the cost of delivered reasoning from the cost of all attempts, then illustrates the evaluation prompt and stage transitions on one item. Its example shows how target-language delivery can improve even when earlier stages fail through English pivots or looping.

C.4 Efficiency

  • Efficiency is reported under four conventions: adjusted or raw token counts, each with either all attempts or only delivered solutions charged.The shared unit is joint successes per 1,000 tokens relative to the same endpoint's English; adjustment removes language-specific encoding effects, while all-attempt accounting includes undelivered responses.Source: §C.4 · p. 16

D.1 Forced-target-language evaluation prompt

  • The evaluation prompt translates the instruction, problem, and answer-format requirement in full, keeps the literal token Answer: constant, and applies the chat template with thinking enabled.The instruction asks for step-by-step reasoning entirely in the requested language and an answer on its own line after Answer:.Source: §D.1 · p. 17

C.4 Efficiency

  • At the correctness-gated Qwen3-4B endpoint, end-to-end efficiency is near parity in adjusted tokens, whereas raw-token end-to-end efficiency is 87% of English; among delivered solutions, adjusted and raw efficiencies are 117% and 100%.The end-to-end gap is carried by yield: at the released Qwen3-4B checkpoint, Spanish, French, and Arabic convert 0.0%–0.1% of generated tokens into a delivered solution, versus 59.7% in English.Source: §C.4 · p. 16, Figure S2 · p. 18

D.2 The same problem in six languages

  • On the selected item, released Spanish, French, and Arabic traces answer correctly in English, while SFT restores target-language traces but French and Arabic loop; the gated endpoint delivers all five in 536–929 tokens.The Arabic SFT trace repeats one sentence 442 times and the French trace repeats one 25 times; these selected pages illustrate transitions but cannot establish their rates.Source: §D.2 · p. 18, Figure S3 · p. 19, Figure S4 · p. 20

Related papers

Loading 2609.37104v1…