Source-linked AI summary
A Survey of Multilingual Reasoning in Language Models
Akash Ghosh, Debayan Datta, Sriparna Saha, Chirag Agarwal
TL;DR
Multilingual reasoning is an emerging research area because LLMs must combine logical inference with multilingual capability while addressing alignment, bias, and low-resource challenges. This survey systematically reviews methods, training resources, benchmarks, and their reported performance. It finds substantial coverage gaps across languages and domains, while identifying fine-tuning, alignment, prompting, and representation-learning directions for future work.
Problem
Multilingual reasoning remains nascent, with challenges including cross-lingual alignment, low-resource language coverage, cultural adaptation, and limited benchmark representation.
Method
The survey reviews multilingual reasoning methods, training corpora, evaluation benchmarks, and state-of-the-art performance across standard reasoning tasks.
Results
Current benchmarks concentrate on a narrow set of languages, domains, and tasks, while existing methods include prompting, fine-tuning, representation learning, alignment, and model editing.
Takeaways & Limitations
Advancing multilingual reasoning requires broader low-resource and domain coverage alongside scalable alignment and reasoning techniques.
Takeaways & Limitations
Because the field is still early, the survey may omit relevant work and does not provide definitive solutions for improving multilingual reasoning.
Abstract
from arXiv · showhide
While reasoning and multilingual capabilities in language models (LMs) have achieved remarkable progress in recent years, their integration into a unified paradigm - multilingual reasoning - is at a nascent stage. Multilingual reasoning requires language models to handle logical reasoning across languages while addressing misalignment, biases, and challenges in low-resource settings. This survey provides the first in-depth review of multilingual reasoning in LMs. In this survey, we provide a systematic overview of existing methods that leverage LMs for multilingual reasoning, specifically outlining the challenges, motivations, and foundational aspects of applying language models to reason across diverse languages. We provide an overview of the standard data resources used for training multilingual reasoning in LMs and the evaluation benchmarks employed to assess their multilingual capabilities. Next, we analyze various state-of-the-art methods and their performance on these benchmarks. Finally, we explore future research opportunities to improve multilingual reasoning in LMs, focusing on enhancing their ability to handle diverse languages and complex reasoning tasks. Rapid growth of evolving developments in this field can be actively tracked on our project page: [https://github.com/AkashGhosh/Survey-of-Multilingual-Reasoning-in-Language-Models](https://github.com/AkashGhosh/Survey-of-Multilingual-Reasoning-in-Language-Models)
1 Introduction
Multilingual reasoning remains underexplored because multilingual LLMs optimized for fluent generation often struggle with complex inference, especially in low-resource languages. The survey systematically reviews methods, resources, benchmarks, and state-of-the-art techniques for addressing this gap.
- Multilingual LLMs frequently struggle with multi-step logic and nuanced understanding because their training emphasizes next-word prediction rather than logical inference.
- Reasoning research has advanced through Chain-of-Thought, fine-tuning, and hybrid modeling, but has focused mainly on high-resource languages.
- The survey reviews methods, training corpora, evaluation benchmarks, and state-of-the-art techniques for multilingual reasoning across diverse languages.
- Multilingual reasoning is motivated by the need for AI systems that operate across diverse linguistic and cultural contexts, including education and healthcare.
2 Multilingual Reasoning in LLMs
Multilingual reasoning extends LLM reasoning across languages while requiring consistency, low-resource adaptability, cultural contextualization, and cross-lingual alignment. The survey organizes current improvement strategies around prompting, representation learning, fine-tuning, and model editing.
- Preliminaries: LLMs model token-sequence probabilities, whereas reasoning maps premises to conclusions through logical inference strategies such as deduction, induction, abduction, analogy, and commonsense reasoning.
- Desiderata in Multilingual Reasoning: A multilingual reasoning model should produce equivalent conclusions for semantically equivalent premises across languages and generalize to low-resource languages through cross-lingual transfer.
- Research Taxonomy: Current improvement strategies span representation learning, fine-tuning, prompting, and model editing, evolving beyond initial reliance on naive prompting.
- Desiderata in Multilingual Reasoning: Multilingual reasoning also requires culturally contextualized conclusions and aligned reasoning processes across typologically diverse languages.
3 Multilingual Reasoning Datasets
Multilingual reasoning datasets and benchmarks cover multiple tasks, domains, and languages, but remain concentrated in high-resource languages and a narrow set of reasoning areas. The survey therefore emphasizes broader benchmark coverage across underrepresented languages, domains, and tasks.
- Training Corpora: Training resources support mathematical, logical, coding, translation, and multilingual representation learning, with datasets selected according to the target reasoning domain and language.
- Languages Covered: Current datasets predominantly cover Chinese, English, French, and German, producing a long-tail language distribution that leaves many languages poorly represented.
- Benchmark Design: Reliable evaluation requires task- and domain-specific benchmarks because mathematical, causal, commonsense, and typological reasoning impose different challenges.
- Domains and Tasks Covered: Math, legal, and commonsense reasoning dominate multilingual benchmarks at 54%, while science, ethics, visual, tabular, and temporal reasoning remain underrepresented.
- Domains and Tasks Covered: Finance and healthcare still lack dedicated multilingual reasoning benchmarks, exposing a significant gap in evaluation coverage.
- Languages Covered: Benchmarks include high-resource languages most often, while Swahili, Haitian, Nepali, Kannada, Gujarati, and Quechua receive limited or rare coverage.
4 Methods
The survey organizes multilingual reasoning methods into representation alignment, fine-tuning, prompting, and model editing, which address cross-lingual consistency through complementary interventions.
- Four method families are identified: representation alignment, fine-tuning, prompting, and model editing.
- Representation Alignment: Representation alignment seeks similar embeddings for equivalent concepts, reducing inconsistencies in multilingual inference and improving generalization.
- Finetuning: Fine-tuning ranges from small high-quality bilingual datasets to parameter-efficient selective layer tuning that reduces computational demands.
- Prompting: Prompting supports dynamic language adaptation and cross-lingual consistency, including keyword-level English insertion without fully translating non-English prompts.
- Model Editing: Model editing updates multilingual knowledge for specific input-output pairs while aiming to preserve unrelated model behavior and improve multi-hop reasoning.
5 Evaluation Metrics and Benchmarks
The survey evaluates multilingual reasoning through accuracy, reasoning consistency, translation, alignment, and benchmark-specific analyses across mathematical, commonsense, and logical tasks.
- Evaluation Metrics: Evaluation must capture inference errors, translation drift, and reasoning stability across languages rather than traditional NLP performance alone.
- Evaluation Metrics: Core metrics include general and zero-shot accuracy, reasoning accuracy, path consistency, translation success rate, and cross-lingual consistency.
- Evaluation Metrics: Perplexity-based alignment uses token-prediction probabilities, while semantic alignment measures cosine similarity between sentence embeddings across languages.
- Benchmarks: The survey examines MGSM, MSVAMP, xCSQA, and xNLI, covering multilingual mathematical, commonsense, and logical reasoning.
- Benchmarks: MGSM contains 250 translated problems in ten languages, while MSVAMP contains 10k problems across ten languages and tests out-of-domain mathematical generalization.
- Performance on Benchmarks: Across benchmark trends, stronger fine-tuning and preference optimization are described as outperforming weaker alternatives on several multilingual reasoning tasks.
6 Future Directions
The survey calls for collective efforts to reduce bias toward low-resource languages and advance multilingual reasoning research beyond current capabilities.
- The proposed agenda is motivated by the rapid development of reasoning models and the need to extend their benefits across languages.
- The survey urges the community to ensure that reasoning models remain unbiased toward low-resource languages.
- Future work should address multilingual reasoning challenges through coordinated research directions identified by the survey.
1. Multilingual Alignment and Reasoning Trans-
Future research targets multilingual reasoning limitations involving low-resource data, faithfulness, evaluation, multimodality, domain coverage, efficiency, and compositional reasoning.
- Multilingual Alignment and Reasoning Transfer: Cross-lingual transfer, back-translation, and reasoning-focused data augmentation are proposed to address scarce data in different languages.
- Explainable and Interpretable Reasoning: Multilingual causal reasoning and CoT frameworks are proposed to improve faithfulness and interpretability amid translation ambiguities and reasoning inconsistencies.
- Unified Evaluation Metrics: A unified evaluation framework should measure logical consistency, cultural adaptability, and robustness in real-world and adversarial multilingual settings.
- Multimodal Multilingual Reasoning: Multimodal multilingual reasoning remains largely unexplored across tables, text, images, audio, and video, especially for low-resource languages.
- New Benchmarks: New domain-specific benchmarks are needed for high-stakes fields such as healthcare, law, and finance, where accuracy affects decisions.
- Efficient Reasoning Models: Efficient multilingual reasoning models remain underexplored, while post-training reinforcement learning shows mixed results depending on the base model and linguistic diversity.
- Miscellaneous Tasks: Models still struggle with compositional reasoning, long-context reasoning in low-resource languages, and reliable integration of retrieved information.
7 Conclusion
Multilingual reasoning is rapidly evolving around cross-lingual alignment, low-resource language gaps, and cultural adaptation. The survey reviews progress while identifying scalability and domain-specific application gaps.
- The survey identifies cross-lingual alignment, low-resource language gaps, and cultural adaptation as central challenges in multilingual reasoning.
- It highlights advances in fine-tuning, prompting, and representation learning for multilingual reasoning.
- The survey identifies scalability and domain-specific applications as remaining gaps.
- It calls for advanced alignment techniques, culturally aware reasoning, and scalable architectures.
8 Limitations
The survey is an early systematic account of multilingual reasoning, but it may omit relevant work and does not claim definitive solutions. Its goal is instead to analyze existing approaches and compare their performance on current benchmarks.
- As the first survey dedicated to multilingual reasoning, it may unintentionally omit some relevant studies.
- Because the field remains in its early stages, the survey does not aim to provide definitive solutions for improving multilingual reasoning.
- The survey instead analyzes existing approaches and evaluates which techniques perform more strongly across current benchmarks.
A Appendix
The appendix positions this work as the first survey focused specifically on multilingual reasoning and situates it among broader multilingual-LLM and reasoning surveys. It also documents the languages represented across reasoning datasets.
- Earlier surveys cover multilingual-LLM resources, taxonomies, corpora, alignment, architectures, evaluation, and applications.
- Unlike general reasoning surveys, this survey focuses specifically on logical inference across languages, including misalignment, bias, low-resource gaps, and tailored benchmarks.
- The appendix provides a tabular overview of languages used in different reasoning datasets.
- Table 1 maps language codes to their corresponding languages.
Distribution of papers covering different aspects of Reasoning
The appendix catalogs multilingual reasoning datasets across mathematics, coding, common sense, legal, moral, compositional, and temporal domains. Their language coverage ranges from small multilingual sets to broad collections spanning many languages.
- Table 2 organizes multilingual datasets by paper, domain, and language coverage.
- The catalog spans mathematics, coding, common sense, legal, moral, compositional, and temporal reasoning datasets.
- Other datasets target narrower multilingual settings, including IndiMathQA, CRUXEval, MCR, mTEMPREASON, and mARC.
- MMWP Benchmark covers mathematics across a broad set of languages, while mMMLU covers common sense across numerous languages.