PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners

Yihong Liu, Raoyuan Zhao, Hinrich Schütze, Michael A. Hedderich

arXiv:2601.02996v2cs.CL

TL;DR

The paper addresses the limited evidence about how latent reasoning behaves across languages. It uses truncated reasoning traces and representational analyses across 11 languages to measure early answer formation and compare internal dynamics. The results show real but uneven multilingual latent reasoning, with stronger signals in resource-rich languages and an English-centered shared pathway, while harder tasks largely suppress observable early formation.

  • Problem

    Existing latent-reasoning evidence focuses mainly on English, leaving multilingual prevalence and cross-language internal pathways unresolved.

  • Method

    The study uses truncated reasoning traces and representational analyses across two mathematical benchmarks and 11 languages to measure early answer formation and internal prediction dynamics.

  • Results

    Latent reasoning exists across languages but is stronger in resource-rich languages, weaker on harder tasks, and internally converges toward an English-centered pathway.

  • Takeaways & Limitations

    Current LRMs exhibit real but fragile multilingual latent reasoning shaped by English-centric post-training and task complexity.

  • Takeaways & Limitations

    The work is primarily diagnostic and does not propose a new inference method.

Abstract

from arXiv · show

Large reasoning models (LRMs) achieve strong performance on mathematical reasoning tasks, often attributed to their capability to generate explicit chain-of-thought (CoT) explanations. However, recent work shows that LRMs often arrive at the correct answer before completing these textual reasoning steps, indicating the presence of latent reasoning -- internal, non-verbal computation encoded in hidden states. While this phenomenon has been explored in English, its multilingual behavior remains largely unknown. In this paper, we conduct a systematic investigation of multilingual latent reasoning in LRMs across 11 languages. Using a truncation-based strategy, we examine how the correct answer emerges as the model is given only partial reasoning traces, allowing us to measure stepwise latent prediction formation. Our results reveal clear evidence of multilingual latent reasoning, though unevenly: strong in resource-rich languages, weaker in low-resource ones, and broadly less observable on harder benchmarks. To understand whether these differences reflect distinct internal mechanisms, we further perform representational analyses. Despite surface-level disparities, we find that the internal evolution of predictions is highly consistent across languages and broadly aligns with English -- a pattern suggesting an English-centered latent reasoning pathway.

1 Introduction

The paper asks whether latent reasoning occurs across languages and whether languages share internal reasoning pathways. It investigates these questions with truncated reasoning traces and finds multilingual but uneven latent reasoning, alongside shared dynamics converging toward an English-centered pathway.

  • Motivation: Prior work shows that models can compute answers in hidden states before completing explicit chain-of-thought reasoning.Latent reasoning has been observed in factual recall and in models that internally form solutions before articulating them.
  • Research questions: Existing latent-reasoning studies focus almost exclusively on English, motivating questions about multilingual prevalence and cross-language internal mechanisms.The paper asks whether latent reasoning varies in strength across languages and whether languages follow distinct or shared pathways.
  • Approach: The study evaluates two mathematical reasoning benchmarks across 11 languages using truncated traces to measure stepwise early predictions and aggregate latent-reasoning metrics.Representational analysis with the logit lens examines the internal evolution of answer formation.
  • Findings: Latent reasoning exists across languages, but resource-rich languages show stronger early-emergent correctness than low-resource languages.This finding establishes multilingual latent reasoning while showing substantial variation by language-resource level.
  • Findings: Latent reasoning becomes less pronounced as task difficulty increases, with early answer formation largely disappearing on harder benchmarks.The pattern holds across languages and model sizes.
  • Findings: Internal latent-reasoning dynamics are shared across languages and converge toward an English-centered pathway, especially for high-resource languages and correctly solved instances.The paper also reports partial memorization, but latent reasoning remains evident in high-resource languages and scales with model size.

2 Related Work

Prior work distinguishes latent computation from explicit textual reasoning but has studied multilingual behavior only sparsely. This paper addresses that gap by systematically comparing latent reasoning across languages.

  • Multilingual Reasoning: Multilingual reasoning research has primarily examined explicit behavior, while this work targets latent reasoning across languages.The paper positions its contribution as a systematic investigation of an underexplored aspect of multilingual reasoning.
  • Multilingual Reasoning: Explicit multilingual reasoning gaps have been linked to uneven language-specific traces and failures in understanding low-resource inputs.Prior work also reports strong language bias and the usefulness of translate-then-solve strategies for under-resourced languages.
  • Implicit Latent Reasoning: Latent reasoning denotes internal computation in hidden representations rather than step-by-step textual explanations.Prior studies describe parallel latent paths, increasing confidence during explicit reasoning, and continued generation after an answer is internally formed.
  • Implicit Latent Reasoning: Existing latent-reasoning methods and studies focus almost exclusively on English, leaving cross-language emergence and variation unresolved.The present work responds by evaluating and comparing latent reasoning dynamics across languages.

3 Experimental Setup

The study uses distilled DeepSeek-R1 models, two multilingual mathematical benchmarks, 11 languages grouped by resource level, and prompts that steer reasoning into the input language. This setup enables comparisons of latent reasoning across model capacity, task difficulty, and language-resource conditions.

  • Models: The experiments use three DeepSeek-R1-Distill-Qwen models with 7B, 14B, and 32B parameters to study capacity effects.The variants are selected for strong reasoning performance and multiple model sizes.
  • Benchmarks: MGSM contains 250 grade-school math problems translated from English into 10 additional languages, keeping the underlying problems identical across languages.This design supports multilingual comparisons of reasoning dynamics.
  • Benchmarks: Multilingual AIME uses translated AIME2024 and AIME2025 problems in the same 11 languages to examine latent reasoning on substantially harder competition-level mathematics.Its greater difficulty complements MGSM's grade-school problems.
  • Language groups: Languages are grouped as high-resource, mid-resource, or low-resource according to training-resource availability and coverage in multilingual language models.The groups are high-resource EN, ES, DE, FR, RU, ZH; mid-resource BN, JA, TH; and low-resource SW, TE.
  • Prompting: A language-specific prefix inserted after the <think> token steers explicit reasoning traces into the same language as the prompt.The strategy addresses models' tendency to generate traces in a different language.

4 Latent Reasoning Identification

The paper identifies latent reasoning by testing answer predictions from truncated reasoning traces and separating correctness from visible answer articulation. Results show stronger latent reasoning in high-resource languages and on MGSM than in low-resource languages or Multilingual AIME.

  • Truncating Reasoning Traces: Truncated pass@k measures whether correct answers emerge from partial reasoning traces, with high accuracy at small truncation ratios indicating possible latent reasoning.The protocol retains selected fractions of each trace and elicits numerical predictions from the original problem plus the retained prefix.
  • Evaluation Metrics: Gold-in-trace rate distinguishes correctness after visible answer articulation from correctness before the answer appears in the truncated trace.At small truncation ratios, low gold-in-trace rates make latent reasoning more plausible, whereas high rates suggest explicit answer articulation may support correctness.
  • Evaluation Metrics: AUTC measures how early and robustly correct predictions emerge, while AUGC tracks when gold answers become articulated in the trace.Higher AUTC corresponds to accuracy reached with shorter trace prefixes; higher AUGC indicates earlier answer articulation.
  • Evaluation Metrics: LRS downweights correctness after the answer is visible and upweights correctness before visibility, serving as a proxy for latent reasoning capability.The metric combines truncated performance with the complement of the gold-in-trace rate.
  • Results and Discussion: On MGSM, early correct predictions are largely absent from the visible trace, supporting latent reasoning before explicit verbalization.High-resource languages show nontrivial pass@1 accuracy at zero reasoning steps, including around 0.2 for English, French, and Chinese.
  • Results and Discussion: Latent reasoning is resource-dependent and difficulty-sensitive: high-resource languages outperform low-resource languages, while LRS falls from about 0.38 to 0.03 for English across MGSM and Multilingual AIME.Larger models improve AUTC and LRS but do not remove language-resource gaps; latent reasoning is largely undetectable on the harder benchmark.

5 Latent State Dynamics

Across languages, latent answer formation follows highly similar layer-wise dynamics, while representational alignment with English varies by resource level and correctness. These results support a largely shared, English-centered latent reasoning pathway, with typological relatedness as a possible confound.

  • 5.1 Dynamic of Ranking Across Layers: Highly similar rank trajectories across languages suggest a largely language-invariant mechanism for forming solutions across layers.The comparison covers fixed models across languages and datasets.
  • 5.1 Dynamic of Ranking Across Layers: Larger models appear to distribute reasoning more evenly across depth, allowing intermediate representations to accumulate evidence toward the correct solution.The authors relate this pattern to stronger representation compression and greater capacity to exploit shared conceptual spaces.
  • 5.2 Hidden State Similarity: High-resource languages show consistently higher hidden-state similarity to English across layers and reasoning steps, including Chinese and Russian.Japanese and Telugu exhibit lower similarity to English.
  • 5.2 Hidden State Similarity: Similarity to English alone cannot distinguish an English-centered reasoning process from alignment caused by shared correct answers.The analysis therefore compares correct and incorrect examples and similarity with English versus other languages.
  • 5.2 Hidden State Similarity: High-resource languages retain strong English alignment across correct and incorrect examples, whereas low-resource languages show weaker English alignment and relatively greater similarity to other languages.Mid-resource languages occupy an intermediate regime, with stronger English alignment for correct instances.
  • 5.2 Hidden State Similarity: Crosslingual similarity patterns may partly reflect linguistic or typological relatedness between languages.This is identified as a potential concern when interpreting the observed alignment.

6 Is Latent Reasoning Behavior Specific to Distilled Models?

The study tests whether multilingual latent reasoning depends on distilled models by comparing distilled, instruction-tuned, and RL-trained models with common truncation-based metrics. The main multilingual pattern persists across post-training paradigms, while RL training is associated with earlier answer articulation and extended reasoning.

  • 6.1 Experimental Setup: High AUTC across models indicates that correct answers can often be produced from partial reasoning traces.The evaluation uses AUTC, AUGC, and LRS to characterize answer emergence and dependence on explicit articulation.
  • 6.2 Results and Discussion: Uneven latent reasoning across languages persists in instruction-tuned and RL-based models, so the disparity is not specific to distilled models.The comparison includes Qwen2.5-32B-Instruct and Nemotron-32B alongside R1-Qwen-32B.
  • 6.2 Results and Discussion: Nemotron-32B achieves substantially higher AUTC but also much higher AUGC and relatively low LRS, indicating frequent early answer articulation.The model often produces the correct answer within the first ∼20% of steps before continuing with reflective reasoning.
  • 6.2 Results and Discussion: On MGSM, models often answer with minimal or no reasoning trace, whereas Multilingual AIME typically requires explicit answer articulation.The harder benchmark shows lower AUTC/LRS and higher AUGC, supporting a central role for task difficulty.

7 Complementary Analysis: Memorization or Latent Reasoning

The complementary analysis tests whether correct answers without visible reasoning reflect memorization or latent reasoning. Perturbation and paraphrase results indicate partial memorization but substantial solution recomputation and reasoning robustness.

  • Method: The analysis focuses on MGSM questions answered correctly without a reasoning trace, where memorization and latent reasoning are especially difficult to distinguish.The study applies controlled edits to these ambiguous cases.
  • Method: NumEdit changes one number so the original gold answer becomes incorrect, measuring sensitivity through the matching ratio with that original answer.Lower matching ratios indicate greater sensitivity to the perturbation.
  • Method: Paraphrase reorders and rewrites the question while preserving its numerical content and semantics, so unchanged-answer performance is measured with standard accuracy.The gold answer remains unchanged under this edit.
  • Results: NumEdit matching ratios are typically around 30% without traces, but fall below 25% for smaller models and below 20% for the 32B model when new traces are generated.High-resource languages generally show lower matching ratios than low-resource languages in the without-trace setting.
  • Interpretation: Together, the results indicate partial memorization but show that models often recompute solutions and do not rely solely on surface-level question-wording patterns.The evidence combines lower matching after numerical edits with robustness to paraphrasing.
  • Results: Paraphrase accuracy is typically above 70% without traces and improves further with new traces, reaching near-perfect accuracy for R1-Qwen-32B in high-resource languages.Performance is lower in under-resourced languages, but the same improvement trend holds.

8 Conclusion

The paper finds that multilingual latent reasoning is real but uneven, and that its internal dynamics are broadly shared across languages through an English-centered pathway. These findings remain consistent with evidence that the behavior cannot be explained by surface memorization alone.

  • Conclusion: Truncation analyses show latent reasoning is strong in resource-rich languages on easier tasks, weaker in low-resource languages, and largely undetectable on harder benchmarks.The conclusion characterizes the capability as highly uneven across language resources and task complexity.
  • Conclusion: Representational analyses reveal consistent layer-wise dynamics across languages that converge toward an English-centered pathway, especially for high-resource languages and correctly solved instances.The conclusion frames these dynamics as shared despite surface-level differences.
  • Conclusion: The findings suggest that current LRMs exhibit real but fragile multilingual latent reasoning shaped by English-centric post-training and task complexity.The conclusion also states that the observed behaviors cannot be explained by surface-level memorization alone.

Limitations

The study’s main limitations concern truncation granularity, answer-detection reliability, benchmark scope, model scale, and its diagnostic rather than improvement-oriented focus.

  • Measurement: Step-level rather than token-level truncation improves interpretability but may miss finer-grained dynamics of latent answer formation.Token-level analysis is left for future work.
  • Measurement: Gold-in-Trace uses string matching, which can produce false positives when an intermediate value happens to equal the final answer.The authors note this effect is limited in their setting but suggest more robust matching strategies.
  • Scope: Experiments focus on mathematical reasoning benchmarks, which may limit generalizability to domains such as commonsense reasoning.Extending the analysis beyond mathematics is identified as future work.
  • Scope: Computational constraints limit experiments to models with up to 32B parameters, leaving larger-scale models for future investigation.The authors report consistent trends across the included models and post-training paradigms.
  • Contribution: The work is primarily diagnostic and does not propose a new inference method for converting its multilingual reasoning insights into concrete improvements.The authors identify translating these insights into improved multilingual inference as future work.

Ethical Considerations

The supplied passages describe methodological details and metric safeguards, but do not provide substantive ethical considerations. The reported analysis concerns reasoning traces, perturbation tests, and false-positive control.

  • Methodological context: Table 4 reports average and median reasoning-step counts for MGSM and Multilingual AIME across languages.These statistics inform the truncation granularity used for each benchmark.
  • Methodological context: MGSM typically has about 10 reasoning steps per language, motivating a 10% truncation granularity that roughly removes one step at a time.The passage presents this as the basis for the MGSM truncation setting.
  • Methodological context: Multilingual AIME has longer average traces but markedly smaller medians because a small number of outliers affect its limited 60-problem dataset.The study therefore uses 5% truncation granularity for Multilingual AIME to account for crosslingual variation while controlling computational cost.
  • Metric caveat: Gold-in-Trace may falsely match an intermediate value to the final answer, although multi-digit answers and conditioning on correct predictions reduce this risk.The authors therefore regard the metric as a reasonable proxy for early answer articulation in this setting.

C Complete Truncation Results

Truncation results show that MGSM predictions often become correct before the gold answer is explicitly articulated, whereas Multilingual AIME relies more on later reasoning steps. High-resource languages generally show earlier and more stable gains than mid- and low-resource languages.

  • Truncation patterns: On MGSM, accuracy increases before the gold answer is explicitly articulated, while Multilingual AIME accuracy typically remains low until late truncation ratios.This contrast indicates stronger early answer formation on MGSM than on the harder Multilingual AIME benchmark.
  • Crosslingual variation: High-resource languages generally achieve earlier and more stable gains than mid- and low-resource languages across the truncation curves.
  • Causal decomposition: On MGSM, early and intermediate accuracy gains are largely attributed to answers not yet appearing in the truncated trace, indicating latent reasoning.The causal decomposition separates gains from newly articulated answers, previously visible answers, and answers absent from the visible trace.
  • Causal decomposition: On Multilingual AIME, gains are sparser and increasingly dominated by explicit answer articulation or answers already present in the visible trace.This pattern reflects stronger dependence on explicit reasoning and a reduced role for early latent solution formation.

D Complete Similarity results

Representational analyses compare hidden states across languages and with English while complementary truncation and perturbation analyses probe latent reasoning and memorization. High-resource languages align more strongly with English representations, while controlled perturbations validate that numerical changes alter answers and paraphrases preserve them.

  • Crosslingual representational alignment: Hidden states are compared with English at each truncation ratio and aggregated across layers and reasoning steps.The analysis uses the final token of each partial reasoning trace for each instance.
  • Crosslingual representational alignment: High-resource languages exhibit stronger alignment with English representations across both benchmarks, whereas mid- and low-resource languages show reduced alignment.
  • Correctness and alignment: Across models, high-resource languages remain strongly similar to English regardless of correctness, while low-resource languages can be more similar to other non-English languages.The comparison uses correctly and incorrectly solved MGSM examples and includes R1-Qwen-7B, 14B, and 32B.
  • Perturbation probes: NumEdit and Paraphrase are applied to correctly solved MGSM instances at 0% truncation to distinguish memorization from reasoning.NumEdit changes one number, whereas Paraphrase preserves meaning.
  • Perturbation solvability: NumEdit matching is typically below 10%, while Paraphrase matching is often above 95% across models and languages.These trends support that numerical perturbations change the target answer and meaning-preserving rewrites retain solution consistency.
Loading 2601.02996v2…