PaperlayerView exact revision on arXiv

Source-linked AI summary

Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning

Mehrnaz Mofakhami, Ananya Sahu, Alejandro R. Salamanca, Daniel D'souza, Alexandre Berard, Thomas Euyang, Marzieh Fadaee, Julia Kreutzer

arXiv:2609.10445v1cs.CL

TL;DR

Multilingual reasoning models often switch to English, limiting in-language reasoning despite understanding non-English prompts. This paper optimizes multilingual SFT data composition and scheduling, finding that broad language coverage, multilingual non-reasoning data, and sufficient English reasoning supervision support reasoning transfer across languages.

  • Problem

    Reasoning models remain overwhelmingly English-centric, leaving limited evidence on how to produce reasoning traces that match multilingual prompts.

    Source: §1 · p. 2

  • Method

    The paper optimizes multilingual SFT through translated reasoning traces, broader language coverage, multilingual non-reasoning data, and English reasoning supervision.

    Source: §1 · p. 3, §7 · p. 21

  • Results

    Broader language coverage improves transfer without interference, while multilingual non-reasoning data benefits both in-language reasoning and cross-lingual transfer.

    Source: §1 · p. 4

  • Takeaways & Limitations

    Reasoning capability and reasoning language can be learned through partly separable supervision, making reasoning language transferable rather than requiring language-by-language construction.

    Source: §7 · p. 21

    Multilingual reasoning supervision relies on translated English traces, which can anchor reasoning style and cultural framing to English and degrade for low-resource languages.

    Source: §8 · p. 21

Abstract

from arXiv · show

Reasoning language models have made substantial advances on a variety of complex tasks, yet their capabilities remain overwhelmingly English-centric: models primarily reason in English regardless of the language they are prompted in. This is inaccessible for non-English-speaking users, risks losing the intent of the original question, and forgoes knowledge more readily expressed in the target language. In this work, we advance L2 reasoning, the ability of a model to reason consistently in the language of the user's prompt, thus building an in-language bridge between the prompt and the answer. We approach this problem from a data-centric angle, investigating how to optimize data composition and scheduling in SFT for reasoning generalization. Building Tiny Aya L2-Thinker at 3.35B scale, we achieve an L2 reasoning rate above 93% across 60 languages on 6 benchmarks spanning math, commonsense reasoning, instruction following, open-ended generation, and cultural reasoning while keeping performance strong. We show the path to generalizing L2 reasoning to held-out languages goes through broader language coverage, readily available multilingual non-reasoning data, and a sufficient English reasoning backbone. These findings indicate that reasoning is a language-agnostic behavior that can be transferred across typologically diverse languages through careful data mixing and without requiring reasoning supervision in every target language. We release our model weights and multilingual reasoning data to support further research on accessible, in-language reasoning.

1 Introduction

The paper addresses the gap between multilingual understanding and L2 reasoning, where models reason in the prompt’s language, through data-centric SFT composition and scheduling. It shows that broad language coverage, multilingual non-reasoning data, English reasoning supervision, and data mixing can improve multilingual accuracy and in-language reasoning without requiring reasoning supervision in every language.

Motivation

  • L2 reasoning closes the gap between multilingual prompting and reasoning, since models commonly answer non-English prompts with English traces that can lose intent, nuance, and culturally embedded knowledge.Native-language reasoning also makes intermediate traces more accessible for inspection and evaluation, including in high-stakes domains.Source: §1 · p. 2

Contribution

  • Tiny Aya L2-Thinker extends a 3.35B multilingual base to dual-mode L2 reasoning across 45 languages, achieving broad coverage with minimal performance loss.The model also shows less degenerate repetition and smaller cross-language variation in L2 reasoning rates than compared models.Source: §1 · p. 3

Data composition

  • Broadening multilingual supervision increases L2 reasoning on languages without direct supervision while maintaining accuracy and covered-language reasoning, outperforming separate regional specialists.These findings indicate that the curse of multilinguality does not apply to reasoning in this setting.Source: §1 · p. 4
  • Multilingual non-reasoning instruction data improves both L2 reasoning and accuracy on unseen languages, while English reasoning remains necessary, especially for difficult mathematics.This makes readily available multilingual instruction data a cheap lever for cross-lingual transfer.Source: §1 · p. 4

Training strategy

  • Data mixing provides better task-accuracy and L2-reasoning trade-offs than sequential adaptation or weight merging.The study therefore treats how supervision is combined as a core design choice alongside language coverage and data type.Source: §1 · p. 4

Conclusion

  • The results suggest reasoning is largely language-agnostic and that training can control the bridge language between a user query and final answer with limited multilingual reasoning data.The authors release model weights and multilingual reasoning data to support accessible in-language reasoning research.Source: §1 · p. 4

2 Methodology

The methodology builds in-language reasoning through SFT by translating abundant English reasoning traces into target languages and jointly training English reasoning, multilingual reasoning, and multilingual instruction-following data.

2 Methodology

  • SFT combines English reasoning teachers, translated L2 reasoning traces, and multiple data sources to develop reasoning beyond easily verifiable domains like math.The method focuses on producing L2 reasoning without requiring large training datasets or separate reasoning systems for every language.Source: §2 · pp. 4–5, §2.1 · p. 5

2.1 Data augmentation via translation

  • Translate-train converts abundant English reasoning traces into multilingual supervision while preserving the original problem and solution structure.Training-time translation permits quality control through translation-model selection and filtering, unlike test-time translation.Source: §2.1 · p. 5
  • Translated reasoning is costly and error-prone because long traces, specialized domains, and low-resource tokenization can degrade logical coherence and factual correctness.The method therefore treats translated reasoning as scarce, focusing on 44 diverse languages with fewer than 5K samples per language.Source: §2.1 · p. 5
  • The translation seed set filters English prompts for consistent language identification and removes code-switching, translation-related content, and constraints difficult to preserve under translation.Excluded constraints include exact word counts, length bounds, and capitalization or formatting requirements.Source: §2.1 · p. 5

2.2 Multi-Task post-training

  • Joint multi-task training combines English reasoning, multilingual reasoning, and multilingual non-reasoning data to teach language alignment in both reasoning and direct-answer modes.Reasoning examples use explicit traces, whereas non-reasoning examples use empty reasoning blocks and direct answers; all sources are mixed within each batch.Source: §2.2 · pp. 5–6

3 Experimental Setup

The experiments start from a 3.35B multilingual Tiny Aya model extended to 32K context, then combine English reasoning, translated multilingual reasoning, and multilingual non-reasoning data. Evaluation spans 60 languages, diverse reasoning and generation capabilities, complementary metrics, and comparisons with language-forced and larger L2-reasoning models.

3.1 Base Model

  • The study uses Tiny Aya Base, a 3.35B model supporting 70+ languages across five world regions, as the common experimental starting point.Its context length is extended from 8K to 32K tokens for the experiments.Source: §3.1 · p. 6

3.2 Long-Context Training

  • The long-context stage resumes cooldown training with a 1.25 × 10−4 initial learning rate, interleaving 8K and 32K sequences at a 3:1 ratio.The mixture is balanced across domains and context-length buckets to preserve diversity and stabilize the transition.Source: §3.2 · p. 6

3.3 Training Data

  • Training combines a simple English reasoning mix for controlled experiments, an extended mix for scaling and the final model, translated multilingual reasoning, and ∼4.9M multilingual non-reasoning samples across 67 languages.The final multilingual reasoning data teaches reasoning in 45 languages including English, while empty thinking blocks let non-reasoning examples preserve direct-response capabilities.Source: §3.3 · pp. 6–7

3.4 Evaluation

  • Evaluation spans math, multicultural and multilingual reasoning, localized instruction following, and linguistically diverse open-ended generation across 60 high- and low-resource languages.Source: §3.4 · p. 8

3.5 Baselines & External Comparisons

  • Comparisons include Qwen3.5-4B with user-message or reasoning-trace language forcing, a data-controlled English-only Tiny Aya model, and larger L2-optimized models.The primary external comparison targets a similarly scaled multilingual model trained only on English reasoning, while Table 1 reports accuracy and L2 reasoning rate.Source: §3.5 · p. 9, Table 1 · p. 10

4 Results

Across six benchmarks, Tiny Aya L2-Thinker achieves above 93% in-language reasoning while outperforming language forcing in L2 rate and maintaining strong accuracy. It also limits repetition and remains effective across increasingly low-resource languages with efficient reasoning traces.

4.1 Forcing the reasoning language is not a substitute for training

  • Tiny Aya L2-Thinker exceeds Qwen3.5-4B with thinking-prefix language forcing in L2 reasoning on every benchmark, reaching above 93% across seen and unseen languages.It surpasses M-Thinker-7B on five of six benchmarks at half the size and exceeds Magistral-Small-24B on all tasks.Source: §4.1 · p. 11

4.2 In-language reasoning comes at minimal cost to accuracy

  • In-language reasoning reduces other-language accuracy by at most 2–3% on five of six tasks, while increasing L2 reasoning from near zero to above 93%.PolyMath is the main exception, dropping from 18.6% for the English-reasoning model to 11.1%; English performance is otherwise largely preserved, with a 5-point GlobalPIQA drop.Source: §4.2 · pp. 11–12

4.3 Long traces signal doomlooping, not deeper reasoning

  • Longer reasoning traces correlate with higher 4-gram repetition, while Tiny Aya L2-Thinker stays concise on easier benchmarks and allocates more tokens to PolyMath.Qwen3.5-4B with thinking-prefix language forcing reaches above 0.6 and 0.4 repetition scores on Marco-Bench-MIF and MIST-OEG, respectively, where its extra tokens do not improve performance.Source: Figure 4 · p. 11, §4.3 · p. 12

4.4 Efficiency and Coverage on Low-resource Languages

  • Tiny Aya L2-Thinker remains strong across all resource tiers: its L2 rate is 99% on tiers 1–2, 94.3% on tier 3, and 94.5% on tier 4.It is the only compared model that stays strong on coverage, accuracy, and efficiency as language resourcedness falls, using under 2,000 thinking tokens on average.Source: §4.4 · p. 13, Figure 5 · p. 13
  • Tiny Aya L2-Thinker covers 45 languages versus M-Thinker-7B’s 6, with roughly half its L2-rate variance, while matching Qwen3.5-4B’s performance using higher L2 rates and more efficient traces.This broader coverage and robustness hold despite Tiny Aya L2-Thinker being half the size of M-Thinker-7B.Source: §4.4 · p. 13

5 Analysis and Building Blocks

The analysis identifies broader language coverage, multilingual non-reasoning data, and a sufficient English reasoning backbone as pillars of L2 reasoning transfer. Across alternatives, joint data mixing provides the best accuracy–L2 reasoning trade-off and predictable English fallback failures.

5 Analysis and Building Blocks

  • The analysis centers on transferring L2 reasoning with minimal target-language reasoning supervision by combining broader coverage, multilingual non-reasoning data, and sufficient English reasoning.These pillars transfer reasoning to languages with little or no in-language reasoning data, while mixing balances task accuracy and L2 reasoning.Source: §5 · p. 13

5.1 More languages improve transfer of reasoning behavior

  • Broader language coverage monotonically improves unseen-language L2 reasoning while preserving English accuracy, seen-language performance, unseen-language accuracy, and seen-language reasoning.Unseen-language L2 reasoning rises 14 →35 →60 on MGSM, 10 →13 →16 on Marco-Bench-MIF, and 19 →21 →42 on GlobalPIQA.Source: §5.1 · pp. 14–15, Figure 6 · p. 14

5.2 A small proportion of non-reasoning data yields positive transfer across metrics

  • Adding 10% multilingual non-reasoning data raises MGSM unseen-language L2 reasoning from 46% to 89% and task accuracy from 49% to 67%.The mechanism is input–output language anchoring that transfers into reasoning mode; the useful range is around 20–30%, after which empty reasoning traces increase.Source: §5.2 · p. 15

5.3 English reasoning data as the backbone for mathematical reasoning

  • A heavier English reasoning backbone increases mathematical task accuracy while keeping L2 reasoning nearly stable, but adds little to open-ended task accuracy and temporarily lowers its L2 reasoning rate.Open-ended L2 reasoning recovers with more English data, attributed to improved general instruction following.Source: §5.3 · p. 16, Figure 8 · p. 16

5.4 Data mixing beats sequential adaptation and merging

  • Data mixing offers the best accuracy–L2 reasoning trade-off, while merging collapses reasoning into English and sequential training sacrifices accuracy and causes language confusion.Mixing also provides a consistent fallback to English reasoning when L2 reasoning fails, and later retrofitting adds no further benefits.Source: §5.4 · p. 17

5.5 Comparing contributions of each data pillar

  • The three pillars jointly expand the accuracy–L2 reasoning frontier: English reasoning alone yields 36.7% task accuracy and 12.8% L2 reasoning, multilingual reasoning raises L2 reasoning to 86.1%, and non-reasoning improves both.The progression is summarized as ER, then MR, then NR, producing the final model configuration.Source: §5.5 · p. 17, Figure 9 · p. 17

6 Related Work

Prior work improves multilingual reasoning by preserving English traces, controlling reasoning language at inference, or training native target-language traces. This work differs by targeting readable monolingual traces through multilingual data mixing, scarce translated long-form reasoning, and broad language coverage.

6.1 Improving English reasoning models on multilingual inputs

  • Methods that improve non-English inputs or transfer reasoning across models raise accuracy, but their intermediate reasoning remains predominantly English rather than user-language reasoning.These approaches include multilingual encoder bridges, layer fusion, layer swapping, and steerable interpolation; they generally do not report reasoning-trace language as a metric.Source: §6.1 · p. 18
  • Mixed-language CoT treats trace language as controllable, but retains English scaffolding or code-switching, whereas this work targets traces readable end to end in the prompt language.Son et al. (2026) and Lin & Jurgens (2026) optimize hybrid traces rather than fully monolingual target-language reasoning.Source: §6.1 · p. 18

6.2 Controlling reasoning language at inference

  • Inference-time language forcing increases requested-language traces but can reduce reasoning accuracy, especially in lower-resource languages, because prompting redirects existing capability rather than instilling sustained in-language reasoning.Wang et al. (2025b) find final-answer compliance exceeds trace compliance, motivating language forcing on an English-only reasoner as the primary baseline.Source: §6.2 · pp. 18–19

6.3 Teaching L2 reasoning

  • Translate-train work changes reasoning language but typically uses short mathematical CoT in a few languages, while this work translates long-form traces across 44 languages and three domains as scarce generalization seeds.The translated traces remain under 5K samples per language; they are mixed with multilingual non-reasoning data rather than used alone, and trace language is evaluated directly.Source: §6.3 · p. 19
  • Recent RL methods directly optimize language consistency, but English-reference similarity is not universally correct, and verifiable-reward RL can collapse lower-resource traces toward English at an accuracy cost.This work instead supervises target-language reasoning directly rather than scoring it against an English counterpart.Source: §6.3 · pp. 19–20
  • Joint multilingual training transfers better than per-region specialists in this work, while parameter-space averaging can cancel language conditioning and collapse traces back to English.These findings extend Yang et al. (2025)’s beneficial multilingual scaling observation from RL to SFT and distinguish the training regime from the objective.Source: §6.3 · p. 20

6.4 Existing reasoning data

  • Open reasoning datasets remain overwhelmingly English-trace-based, so this work releases reasoning data covering 44 languages besides English and three domains.Even localized resources often translate prompts and responses while leaving chain-of-thought in English; only a handful of existing resources supervise target-language reasoning directly.Source: §6.4 · pp. 20–21

7 Conclusion

Multilingual reasoning can be achieved with minimal in-language reasoning supervision by combining a sufficiently large English reasoning dataset with a small amount of multilingual non-reasoning data. The findings frame reasoning capability and reasoning language as separable, enabling reasoning language to transfer across languages, including those without direct reasoning supervision.

Conclusion

  • Pairing a small fraction of non-reasoning data with a sufficiently large English reasoning dataset is crucial for multilingual reasoning despite scarce multilingual reasoning data.The approach addresses the English-centric development of reasoning systems while minimizing in-language reasoning supervision.Source: §7 · p. 21
  • Reasoning capability and reasoning language are separable: capability can come from large-scale reasoning supervision, while multilingual supervision teaches the language of expression.This distinction supports learning multilingual reasoning without building it independently for every language.Source: §7 · p. 21
  • Reasoning language can transfer through limited multilingual reasoning data, broader multilingual instruction, and generalization to languages without direct reasoning supervision.The resulting path is more scalable toward systems that reason in the language of their users.Source: §7 · p. 21

8 Limitations

The approach has three limitations: translated reasoning traces may preserve English-centered reasoning patterns, automatic metrics do not assess reasoning quality directly, and L2 reasoning depends on an explicit prompt trigger.

Dependence on translated reasoning traces

  • Translated reasoning traces anchor reasoning style, cultural framing, and problem decomposition to English templates, with quality and naturalness degrading in low-resource languages.The supervision is translated from English traces rather than produced by multilingual teachers or human annotators, despite translation-quality filtering.Source: §8 · p. 21

Evaluation through automatic proxies

  • Automatic evaluation measures linguistic compliance but does not establish the depth, coherence, fluency, usefulness, or cultural appropriateness of reasoning traces.The study reports L2 reasoning rate and open-ended generation quality without human evaluation of reasoning-trace quality.Source: §8 · p. 21

Prompt sensitivity

  • Without the trained explicit instruction “Think in the same language as the prompt,” the model may revert to English, limiting performance with constrained prompting, ambiguous signals, or mixed-language inputs.Reported L2 reasoning rates rely on prepending this instruction to the user prompt.Source: §8 · p. 21

A Training … B.1 Filtering

The training pipeline extends Tiny Aya Base to 32K-token contexts before supervised fine-tuning, while filtering multilingual reasoning data to reduce translation artifacts and preserve instruction fidelity. SFT then trains Tiny Aya L2-Thinker on a large packed mixture with specified optimization and hardware settings.

A.1 Long Context Extension

  • Tiny Aya Base’s context length was extended from 8K to 32K in a single training stage.The extension was applied to Tiny Aya Base (Salamanca et al., 2026).Source: §A.1 · p. 36
  • The long-context transition used 12,000 continuation-pretraining steps on an 8K/32K sequence mixture with a 3:1 ratio.Longer-context data was evenly distributed across three 8K-token buckets spanning 8K to 32K, with a linear schedule starting at 1.25 × 10−4.Source: §A.1 · p. 36

A.2 SFT

  • Tiny Aya L2-Thinker was fine-tuned from Tiny Aya Base for four epochs over a packed SFT mixture of approximately 15.7B tokens per epoch.Training used a standard next-token crossentropy objective, 32 NVIDIA H100 GPUs, fully sharded data parallelism, and a global batch size of 32.Source: §A.2 · p. 36
  • SFT used Adam with β1 = 0.9, β2 = 0.95, weight decay 0.1, gradient clipping at 1.0, and cosine learning-rate decay.The learning rate warmed up for 10 steps, peaked at 1.25 × 10−4, and annealed to 1.25 × 10−5.Source: §A.2 · p. 36

B.1 Filtering

  • The AM thinking dataset (Ji et al., 2025) was filtered to retain English-consistent prompts, reasoning traces, and responses while removing intra-document code-switching.The filtering targeted sources particularly susceptible to translation artifacts.Source: §B.1 · p. 36
  • Filtering removed trajectories discussing translation or naming target languages, as well as prompts requiring exact counts, length bounds, capitalization, or formatting constraints.Language identification used FastText, with GlotLID as a fallback for languages FastText does not cover.Source: §B.1 · p. 36

B.2 Translated Reasoning Data

The training data includes translated reasoning examples across multiple domains and 44 non-English languages, with controlled experiments capped at 5K samples per language but the final model using all available translated data.

Data composition

  • Translated reasoning data spans Math, Science, and General domains across 44 languages besides English, covering Europe, Asia-Pacific, West Asia, Africa, and South Asia.Table 3 reports example counts, while Figure 3 shows the regional and domain composition.Source: §B.2 · p. 36

Data usage

  • Controlled experiments cap translated samples at 5K per language, whereas the final Tiny Aya L2-Thinker model uses all available translated data.Source: §B.2 · p. 36

C Benchmarking Details · C.1 Language Coverage · D Decoding Settings

The evaluation uses balanced seen/unseen-language subsets, broad benchmark coverage constrained by model and language-identification support, and standardized single-completion decoding with model-specific reasoning delimiters. These choices define the language coverage and implementation conditions for the reported experiments.

C Benchmarking Details

  • Seen languages overlap with the ten training languages, while unseen languages are benchmark languages held out from training and sampled at one or two per region.This regional sampling keeps coverage balanced across Figures 6 and 7.Source: C Benchmarking Details · p. 36
  • Urdu substitutes for unavailable unseen-language candidates in West Asia for MGSM and South Asia for Marco-Bench-MIF, with the complete split reported in Table 4.The substitutions use Urdu as the closest available language in the relevant benchmark coverage.Source: C Benchmarking Details · p. 37

C.1 Language Coverage

  • Table 2 covers benchmark languages supported by Tiny Aya and identifiable through FastText or GlotLID, including all languages used to train M-Thinker-7B.Qwen3.5-4B supports 201 languages, but its specific supported languages are undisclosed, so its coverage may not fully align with the listed set.Source: §C.1 · p. 37
  • PolyMath, Marco-Bench-MIF, MIST-OEG, and Macaron-MCQ cover over 90% of their original benchmark languages, whereas MGSM and GlobalPIQA use diverse high- and low-resource subsets.The latter two benchmarks are evaluated on subsets rather than their full original language sets.Source: §C.1 · p. 37

D Decoding Settings

  • Each example receives one completion, with reasoning traces separated from final answers according to each model’s output convention.The conventions include <|START_THINKING|>…<|END_THINKING|>, <think>…</think>, and [THINK]…[/THINK].Source: D Decoding Settings · p. 37
  • All models are evaluated with a 32K-token context window, while Magistral uses the system preamble recommended in its model card.Tiny Aya models use special thinking delimiters, Qwen3.5-4B and M-Thinker-7B use <think>…</think>, and Magistral uses [THINK]…[/THINK].Source: D Decoding Settings · p. 37

E Benchmark results by language · F Doomlooping score scatter plots

Tables 5–12 report per-language accuracy and L2 reasoning rates across benchmarks, while Figure 10 compares thinking length with doomlooping across languages. The results show task-adaptive reasoning efficiency for Tiny Aya L2-Thinker, contrasting with systematically longer Qwen3.5-4B traces and weaker length–doomlooping coupling for Magistral-Small-24B.

E Benchmark results by language

  • Tables 5–12 report per-language task accuracy and L2 reasoning rates for all Table 1 models, including means and standard deviations across languages.PolyMath results are reported separately for its medium, top, and high levels.Source: E Benchmark results by language · p. 37
  • PolyMath scores in § 4 use the weighted average (2 medium + 4 high + 8 top)/14.Source: E Benchmark results by language · p. 37

F Doomlooping score scatter plots

G Ablations · G.1 Joint mixing gives the best accuracy--L2 trade-off · G.2 Joint mixing fails predictably; sequential scatters

The ablation compares joint mixing, sequential adaptation, and specialist merging under a fixed multilingual training setup. Joint mixing provides the strongest accuracy–L2 reasoning trade-off and a stable English fallback, whereas sequential adaptation scatters fallback languages and merging suppresses in-language reasoning.

G Ablations

  • The ablation reuses a ten-language, five-region setup to compare joint mixing with sequentially adapted English reasoning and merged regional specialists.All-Mixed trains on English reasoning and all ten languages’ L2 data jointly; All-Seq adapts an English-only reasoner sequentially; Spec-mrg averages separately trained specialist weights.Source: G Ablations · p. 39, §G.1 · p. 39

G.1 Joint mixing gives the best accuracy--L2 trade-off

  • Sequential adaptation reaches the prompt language most often on held-out inputs but lowers accuracy by forgetting part of the base English capability.The comparison uses a fixed training budget across All-Mixed, All-Seq, and Spec-mrg.Source: §G.1 · p. 39
  • Merging wins task accuracy but collapses in-language reasoning because averaging specialist weights cancels language conditioning, producing seen-language rates of 59/17/49.The rates are ordered MGSM/Marco-Bench-MIF/GlobalPIQA and contrast with 99/95/100 for mixing.Source: §G.1 · p. 39

G.2 Joint mixing fails predictably; sequential scatters

  • Overall, joint mixing conditions on the input language while retaining English as a predictable fallback, unlike sequential adaptation’s competing language associations.This explains why mixing offers a more predictable unseen-language fallback while preserving the accuracy–L2 trade-off.Source: §G.1 · p. 39, §G.2 · p. 39
  • Sequential adaptation scatters fallback reasoning across languages seen during later fine-tuning, making its thinking language harder to predict, correct, and audit.The competing associations created by sequential adaptation resist prediction and correction.Source: §G.2 · p. 39

G.1 Joint mixing gives the best accuracy--L2 trade-off

  • Mixing is never dominated: it preserves near-ceiling seen-language L2 reasoning, stays within a few accuracy points of merging, and is best on both axes for unseen MGSM languages.Seen-language rates are 99/95/100 for mixing versus 59/17/49 for merging across MGSM/Marco-Bench-MIF/GlobalPIQA.Source: §G.1 · p. 39, Figure 11 · p. 40

G.2 Joint mixing fails predictably; sequential scatters

  • Joint mixing falls back stably to English on unseen languages, while regional specialists and merged models reason in-language less often and primarily in English, respectively.A stable fallback is more auditable because downstream systems can anticipate it.Source: §G.2 · p. 39, Figure 12 · p. 40

H Reasoning Errors

The examples show Tiny Aya L2-Thinker consistently reasons and responds in Korean, while competing models exhibit code-switching, excessive verbosity, or failure to generate. Tiny Aya also achieves higher overall task scores with perfect language consistency, alongside stronger coherence and naturalness in one comparison.

Magistral-Small-24B comparison

  • Tiny Aya L2-Thinker reasons and generates in Korean with perfect language consistency, outperforming Magistral-Small-24B overall while scoring higher on coherence and naturalness.Magistral-Small-24B reasons primarily in English, mixes German, English, and Korean in its answer, and produces a more verbose trace.Source: H Reasoning Errors · p. 49

Qwen3.5-4B comparison

Related papers

Loading 2609.10445v1…