PaperlayerView exact revision on arXiv

Source-linked AI summary

PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages

Daryna Dementieva, Nikolay Babakov, Kathy Hämmerl, Ilseyar Alimova, Jindřich Libovický, Shu Okabe, Miras Baisbay, Lukas Edman, Abrorkhon Inomkhujaev, Antonia Karamolegkou, Mateusz Lango, Volkan Özer, Nikola Selic, Subhankar Swain, Tsedeniya Kinfe Temesgen, Galit Bary Weisberg, Alexander Fraser

arXiv:2607.05992v1cs.CLcs.AI

TL;DR

Existing mathematical reasoning benchmarks largely focus on high-resource languages, limiting evaluation in underrepresented linguistic settings. PluraMath extends PolyMath with human-validated data in 18 underrepresented languages and shows a persistent performance gap relative to high-resource languages, with stronger results associated with instruction-following ability.

  • Problem

    Mathematical reasoning benchmarks remain heavily skewed toward English and other high-resource languages, while reasoning ability does not transfer easily across languages.

  • Method

    PluraMath extends PolyMath to 18 underrepresented languages across 6 language families using automatic translation drafts rigorously checked and corrected by native speakers.

  • Results

    A persistent mathematical reasoning performance gap remains between high-resource and underrepresented languages, with stronger results primarily associated with general instruction-following ability.

  • Takeaways & Limitations

    PluraMath enables multilingual mathematical reasoning evaluation across diverse linguistic conditions and provides evidence that translation capability alone does not explain stronger performance.

  • Takeaways & Limitations

    Benchmark construction depends on minimal machine-translation resources, which may require substantially greater cost and complexity for extremely low-resource languages.

Abstract

from arXiv · show

Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites. The recently released PolyMath (Wang et al., 2025) dataset represents a significant step forward, yet its coverage is still limited to 18 only high-resource languages. To address this gap, we introduce PluraMath, an extension of PolyMath to 18 additional {underrepresented languages spanning 6 language families -- ranging from mid-resource to extreme low-resource settings. We constructed the dataset through a human-curated pipeline, where native speakers thoroughly validated pre-computed translations. Using PluraMath, we then benchmark 27 reasoning LLMs across four model scales -- small, mid-size, large, and closed-source ensembles -- probing the multilingual mathematical reasoning capabilities of state-of-the-art models under diverse linguistic conditions. Our fine-grained analysis confirms a persistent gap in mathematical reasoning performance between high-resource and underrepresented languages, with stronger results largely associated with better instruction-following ability. We fully open-source our dataset, data acquisition pipeline, and evaluation framework, with the goal of lowering the barrier to multilingual benchmark development for underrepresented communities.

1 Introduction

Existing mathematical-reasoning benchmarks and multilingual efforts remain concentrated in English, Chinese, and other high-resource languages, leaving underrepresented languages largely untested. PluraMath addresses this gap by extending PolyMath to 18 underrepresented languages across 6 families with native-speaker validation, open resources, and broad model evaluation.

  • Motivation: Existing mathematical-reasoning benchmarks are predominantly English-only or Chinese-only, while newer multilingual datasets still concentrate on high-resource languages.
  • Motivation: Reasoning-answer misalignment is more severe in non-Latin scripts, and latent reasoning pathways remain English-centred across typologically diverse languages.
  • Benchmark: PluraMath extends PolyMath to 18 underrepresented languages from 6 language families using rigorous native-speaker checks of pre-computed translations.
  • Resources: The authors open-source the complete data acquisition and validation pipeline, including annotator guidelines and quality-control procedures.
  • Evaluation: 27 modern reasoning models are evaluated under three prompting setups across small, mid-sized, large open-weight, and closed-source systems.
  • Findings: Performance remains persistently lower in underrepresented languages; stronger results primarily correlate with instruction-following ability rather than translation capability or high-resource-language prompting.

2 Related Work

Mathematical-reasoning evaluation has been dominated by English-only resources, while newer multilingual benchmarks still primarily cover high-resource languages. Related work also examines cross-lingual reasoning gaps, transfer strategies, and translation-based scaling of evaluation.

  • Benchmark coverage: Early mathematical-reasoning benchmarks were overwhelmingly English-centric, spanning grade-school problems, 14.7B tokens of mathematical web text, and competition-level mathematics.Examples include GSM8K, OpenWebMath, AIME, MathArena, and OMEGA.
  • Benchmark coverage: PolyMath covers 18 languages and four difficulty tiers, but most covered languages are high-resource and already well-served by pretraining corpora.Other multilingual benchmarks, including MGSM8KInstruct and MSVAMP, likewise emerged only recently.
  • Cross-lingual reasoning: Cross-lingual diagnostics find that correct answers can conceal incoherent reasoning, especially in low-resource languages and non-Latin-script outputs.Ovalle et al. report at least twice the reasoning-answer misalignment for non-Latin-script outputs, while Liu et al. find uneven latent reasoning across 11 languages.
  • Cross-lingual reasoning: Cross-lingual transfer methods route reasoning through high-resource pivots or aggregate multilingual traces, with Best-of-L improving MGSM accuracy over single-language reward modeling.MathOctopus translates inputs, while Best-of-L ranks reasoning candidates across languages.
  • Translation-based scaling: Translation has become the dominant strategy for scaling multilingual evaluation, combining machine translation, LLM-based selection, and native-speaker post-editing.LLM translation quality strongly correlates with downstream benchmark performance, although this relationship weakens on reasoning-intensive tasks.

3 PLURAMATH Collection

PluraMath extends PolyMath through a staged translation-and-validation pipeline combining automatic translation, expert native-speaker review, and automated/manual LaTeX checks. Quality requirements and revision effort varied substantially across languages because of differences in language family, linguistic distance, and resource availability.

  • Dataset construction: Each language contains 500 tasks across four difficulty levels, with 125 samples per level.The levels are low, medium, high, and top.
  • Dataset construction: The pipeline uses automatic first-draft translation, native-speaker verification, and automated/manual LaTeX checking with final error analysis.Native speakers checked fluency, terminology, mathematical conventions, and strict LaTeX equivalence to the source.
  • Quality control: Annotators were native speakers with at least Master’s-level Computer Science or Mathematics training, and additional reviewers performed second passes when more checks were needed.In most cases, the primary annotators held PhDs; additional reviewers could hold Bachelor’s or Master’s degrees.
  • Quality control: Nearly half of the languages received a second annotation pass, reflecting substantial variation in required revisions across languages.Revision needs depended largely on language family and the maturity of available NLP and machine-translation technologies.
  • Translation analysis: Hindi, Odia, and Amharic produced notably longer token sequences, while Latin- and Cyrillic-based languages remained closer to high-resource distributions.Lengths were measured with Qwen3-4B base-model tokenization and varied substantially across language families.
  • Translation analysis: Translation problems included LaTeX corruption, incorrect mathematical terminology, morphological and syntactic errors, and residual issues linked to linguistic distance and resource availability.Editing ranged from minimal for Ukrainian, Greek, and Hebrew to extensive for Slovak, Tatar, and Kazakh; specialized language-pair models sometimes outperformed closed-source systems.

4 Reasoning LLMs Benchmarking

The benchmark evaluates 27 reasoning-oriented LLMs across four high-resource and 18 underrepresented languages using three prompting settings and difficulty-weighted accuracy. Results show a persistent performance gap linked to language support, with larger proprietary models more stable but underrepresented-language reasoning less coherent.

  • Benchmark setup: 27 reasoning-oriented LLMs are benchmarked across four high-resource and 18 underrepresented languages, spanning model families, training paradigms, and parameter scales.The models range from sub-billion open-weight systems to frontier proprietary APIs.
  • Benchmark setup: The evaluation compares Base, Base+EN-CoT, and Backtranslated prompting to test whether shifting reasoning into a high-resource language reduces performance gaps.Base uses the target language throughout; Base+EN-CoT requests reasoning in English, while Backtranslated uses NLLB-translated problems and high-resource-language instructions.
  • Math answer correctness: ρ = 0.646 with p = 0.0038 measures the correlation between language-support class and mathematical-reasoning benchmark rankings.This indicates that language support in current language technologies is strongly associated with mathematical reasoning performance.
  • Math answer correctness: +2.15 is the average performance gap between language groups, ranging from +0.67 for Greek and Polish to +4.86 for Chuvash and Ahmaric.Smaller models often perform unstably across language groups, whereas larger recent systems, especially GPT-5.4 and Claude-Haiku-4.5, are considerably more stable.
  • Prompt comparison: Alternative prompting designs yield limited improvements for most models across languages.The benchmark also reports answer-format compliance, reasoning length, and dominant output language alongside aggregated DW-ACC scores.
  • Human evaluation: Reasoning quality declines for underrepresented languages, with less consistent target-language reasoning and less coherent explanations.Models frequently switch to English, often produce disfluent or incoherent reasoning, and frequently fail to finish within the 2k-token limit.

5 Conclusion

PLURAMATH extends four-level mathematical reasoning evaluation to 18 underrepresented languages across six language families, using human-validated benchmark construction. Evaluation of 27 modern reasoning LLMs shows that the performance gap between high-resource and underrepresented languages persists.

  • PLURAMATH extends a four-level mathematical reasoning benchmark to 18 underrepresented languages spanning six language families.
  • The benchmark construction pipeline combines automatic translation for initial drafts with rigorous human evaluation and correction.
  • 27 modern reasoning LLMs were evaluated at scale using PLURAMATH.
  • The performance gap persists between high-resource and underrepresented languages.

Limitations

PluraMath remains limited in language coverage and depends on minimal machine-translation resources, while some evaluations may have uncertain educational relevance. The study also uses a single-generation pass@1 setup and does not investigate continued pretraining or domain adaptation.

  • Coverage and resources: The benchmark covers 18 underrepresented languages but remains far from exhaustive and depends on minimal machine-translation resources.The authors propose their pipeline as a framework for extending reasoning benchmarks to additional digitally represented languages, while noting that many extremely low-resource languages lack such resources.
  • Educational relevance: Advanced mathematical concepts may not be commonly taught in all target-language educational contexts, raising questions about evaluation relevance for some languages.The authors also suggest multilingual mathematical benchmarks could support future access to advanced education in underrepresented languages.
  • Evaluation design: The evaluation uses a single-generation pass@1 setup rather than multiple-generation strategies.
  • Training adaptation: The study does not investigate continued pretraining or domain adaptation on multilingual mathematical corpora.Future work could examine how to collect and balance mathematical training data for underrepresented languages to improve multilingual reasoning performance.

Ethics Statement · A Details and Licensing of Resources

The paper describes measures supporting annotator well-being, planned releases of reasoning artifacts, and limitations concerning computational costs and benchmark contamination. It also provides details, links, and licenses for the resources and LLMs used.

  • Ethics Statement: Annotation was conducted on a non-profit basis by academic researchers and independent contributors, with most contributors acknowledged through co-authorship.The contributors were motivated to support their native languages.
  • Ethics Statement: High-quality machine translations reduced annotator burden by allowing verification and correction instead of translation from scratch.Assignments had flexible two-to-four-week completion windows depending on translation quality.
  • Ethics Statement: Flexible two-to-four-week scheduling produced a manageable workload and positive collaborative environment for annotators.Scheduling depended on the quality of the machine translations provided.
  • Ethics Statement: Evaluation across 22 languages and dozens of LLMs likely incurred substantial computational and environmental costs.The authors plan to release generated reasoning traces and model outputs subject to license compliance.
  • Ethics Statement: The planned release of reasoning traces and model outputs aims to support future research, including distillation into smaller, more accessible models.The release is subject to license compliance and is intended to benefit underrepresented languages.
  • Ethics Statement: The authors do not systematically investigate benchmark contamination, and newer flagship LLMs may have encountered PolyMath after its mid-2025 release.Tasks drawn from mathematics textbooks and olympiad problem sets may also permit incidental memorization across languages.
  • A Details and Licensing of Resources: The resource appendix provides details about the LLMs used in experiments and direct links and licenses for all used resources.This section documents the experimental resources and their licensing information.

A.1 Short Summary of LLMs Families Used for the Experiments … C PLURAMATH Tasks Examples

The experiments span diverse open-weight, multimodal, and closed-source reasoning model families, with resources used under research-compatible licensing and official APIs for closed models. PLURAMATH is prepared through native-speaker translation checks and illustrated with representative mathematical tasks across four difficulty levels.

  • A.1 Short Summary of LLMs Families Used for the Experiments: Qwen checkpoints span 0.8B to 235B-A22B, including dense and Mixture-of-Experts variants, forming a major multilingual reasoning backbone.The included models are Qwen3.5 at 0.8B, 2B, 4B, 9B, 35B-A3B, and 122B-A10B, plus Qwen3-235B-A22B.
  • A.1 Short Summary of LLMs Families Used for the Experiments: Open-weight comparisons also include OpenAI gpt-oss 20B and 120B, Moonshot Kimi-K2.5-Thinking, while closed-source upper bounds use Claude-Haiku-4.5, GPT-5.4, and Gemini-2.5-Flash.The open-weight models are Mixture-of-Experts reasoning systems, and the closed-source models are accessed through provider APIs.
  • A.2 Direct Links and Licensing Information for All Resources Used in This Work: Model and dataset licenses are presented with direct repositories, and closed-source models are accessed exclusively through official APIs under provider terms of service.The stated intended use is academic research on multilingual mathematical reasoning for positive societal impact.
  • B Translations Annotator Instructions: Translation stakeholders recommend an automatic system for benchmark preparation, while native speakers—preferably two—check question texts and preserve mathematical answers and LaTeX.Text outside LaTeX must be translated into appropriate target-language terminology; LaTeX must remain identical to the source, with broken delimiters fixed.
  • B Translations Annotator Instructions: A reasoning proof-of-concept uses the first task at each difficulty level with several popular models to assess comprehension in the target language.Stakeholders run examples through ChatGPT, Gemini, Claude, and Qwen3 variants, then compile a short report.
  • C PLURAMATH Tasks Examples: PLURAMATH examples cover four difficulty levels—low 5, medium 6, high 7, and top 8—alongside their original English PolyMath tasks.Examples include arithmetic, sequence convergence, grid-path enumeration, and an aquaesulian functional-equation problem.

D PLURAMATH and PolyMath Math Tasks Length Comparison … H Detailed Per-Difficulty-Level and Per-Prompting-Ablations Results

The appendices compare multilingual task and reasoning-trace lengths, document prompt and closing-instruction designs, describe hyperparameter and inference-budget procedures, and provide detailed evaluation breakdowns. They report substantial variation in input lengths across language families but broadly similar reasoning lengths across language types.

  • D PLURAMATH and PolyMath Math Tasks Length Comparison: Task lengths vary substantially across language families, with Slavic distributions relatively similar and Hindi, Odia, Amharic, and Turkic languages having considerably longer inputs.Lengths were tokenized with Qwen3-4B, with means and standard deviations reported in Tables 4–7 and box plots in Figure 9.
  • E Prompt Templates and Examples: The experiments use prompt templates with HR as the high-resource pivot language, defaulting to English and substituting German, Russian, or Spanish for non-English baselines.The section provides full prompt text and an illustrative Ukrainian example.
  • E.2 Prompts Example in Ukrainian: Every prompt template is illustrated on a concrete low-difficulty Ukrainian problem, including a Native+EN-CoT format that asks models to reason step by step in English.The example uses a duck-egg word problem and requires the final answer in boxed format.
  • E.3 Per-Language Closing Instructions: The Base prompt keeps inputs monolingual by translating PolyMath’s closing instructions into each target language, with every translation verified by native speakers.All 18 target-language directives are listed in Table 10.
  • F Hyperparameter Search: Hyperparameter selection used two stages: English-only reasoning-effort search followed by a temperature sweep on English, German, and Russian high-resource languages.The second stage included a low-temperature sanity check on English only, without extending it to German or Russian.
  • G Closed Models Inference Budget: The benchmark contains L = 4 difficulty levels, T = 125 tasks each, N = 500 problems, B = 2,000 output tokens per trace, ℓ = 22 languages, and p = 3 prompt designs.Thus, each model produces exactly one million output tokens per language; the full three-design sweep is restricted to selected open-weight models, while remaining large and proprietary models use the base prompt.
  • H Detailed Per-Difficulty-Level and Per-Prompting-Ablations Results: The appendix reports detailed results by difficulty level, reasoning length, and prompting type, with additional visualizations.Appendix H.1 covers per-level results, H.2 reasoning-length comparisons, and H.3 prompting-type results.
  • H.2 Models’ Reasoning Length Comparison Visualization: Reasoning lengths do not differ substantially across language types; underrepresented-language outputs may be slightly longer at lower difficulty but shorter at other levels, while top models show nearly equivalent lengths and variation across groups.Reasoning traces and answers were tokenized with Qwen3-4B and visualized by language and model for aggregated and per-level metrics.

H.1 Per-Difficulty-Level Results · H.3 Per-Prompting-Ablations Results

Performance is strongest on low-difficulty tasks but declines substantially as difficulty rises, with underrepresented languages remaining disadvantaged and only certain larger or proprietary models sustaining non-zero higher-level performance. Prompting ablations show no stable average improvement over base prompting: backtranslation is marginal or ineffective, while English chain-of-thought helps selectively and may reflect extra step-by-step instruction.

  • H.1 Per-Difficulty-Level Results: Most models perform relatively strongly at low difficulty, but accuracy declines substantially at higher difficulty levels.The evaluation covers low, medium, high, and top difficulty levels using answer correctness.
  • H.1 Per-Difficulty-Level Results: Odia, Amharic, and Chuvash remain the most challenging underrepresented languages even at the lowest difficulty level.Aggregated distributions also show lower mean scores for severely underrepresented languages, including on low-difficulty tasks.
  • H.1 Per-Difficulty-Level Results: Only gpt-oss variants, GPT-5.4, and Claude-Haiku-4.5 maintain consistently non-zero performance at higher difficulty levels.Larger and proprietary models are comparatively more stable across language groups than smaller models.
  • H.3 Per-Prompting-Ablations Results: The prompting breakdown compares Base, English chain-of-thought (EnCoT), and backtranslation (Backtr.) strategies exclusively on open-weight models.Base solves entirely in the target language; EnCoT uses English reasoning; Backtr. translates the problem into the corresponding high-resource language.
  • H.3 Per-Prompting-Ablations Results: Backtranslation yields only marginal improvements for LFM and does not improve performance for the remaining systems.The prompting comparison reports DW-Acc averages across high-resource and PLURAMATH target languages.
  • H.3 Per-Prompting-Ablations Results: EnCoT improves performance for some languages in Gemma-3-4b, Nemotron3-Nano-30B, and gpt-oss-120b, but gains may partly reflect added step-by-step instruction.Overall, no substantial stable improvements in average performance are observed over base prompting.

I Human Assessments Results per All Levels · I.1 PLURAMATH Answers and Reasoning Traces Examples

The paper combines a human evaluation of multilingual answers and reasoning traces with Ukrainian examples illustrating model-specific patterns in language use, correctness, fluency, and completion. Results show substantial variation: some models reason in the target language but loop or truncate, while others reason in English, mix languages, or produce incomplete and incoherent traces.

  • I Human Assessments Results per All Levels: Human evaluation covered five models across English, Russian, and 11 PluraMath languages, using annotations of answer correctness, target-language reasoning, soundness, consistency, answer presence, and completion.For each language, annotators assessed 12 samples per level per model, totaling 48 distinct tasks per annotator.
  • I Human Assessments Results per All Levels: Table 16 reports aggregated human-evaluation results by difficulty level, distinguishing high-resource Russian from 10 underrepresented PluraMath languages.Reported cells are percentages of “Yes” among applicable annotations, with Partial scored as 0.5 for Q2–Q3 and NA/blank excluded.
  • I.1 PLURAMATH Answers and Reasoning Traces Examples: The examples section presents model outputs, reasoning traces, and human analyses using Ukrainian instances spanning all five annotated models.The illustrated models are gpt-oss-120b, DeepSeek-V3.2, nemotron3_nano_omni_30ba3b, Ministral-3-8B-Reasoning-2512, and gemma-3-4b.
  • I.1 PLURAMATH Answers and Reasoning Traces Examples: DeepSeek-V3.2 typically reasoned in the target language but frequently entered repetitive self-verification loops, reaching correct answers without properly completing the final response.The example was truncated before a final answer was produced, despite deriving the gold value 70 000.
  • I.1 PLURAMATH Answers and Reasoning Traces Examples: Nemotron3_nano_omni_30ba3b frequently generated non-fluent, mixed-language reasoning, often failed to finish, and sometimes returned apparently random boxed answers.An illustrated mismatch extracted 1 instead of the gold answer 15.
  • I.1 PLURAMATH Answers and Reasoning Traces Examples: Ministral-3-8B-Reasoning-2512 alternated between target-language and primarily English reasoning, with incomplete traces accounting for most incorrect predictions and examples ending in degeneration or truncation.Its shown medium-difficulty example produced no answer and degenerate repetition, while another reasoning trace was cut off during verification.
  • I.1 PLURAMATH Answers and Reasoning Traces Examples: gemma-3-4b consistently reasoned fluently in the target language, although incorrect answers could still contain coherent reasoning with flawed intermediate steps.The accompanying example was truncated without a final answer, despite otherwise fluent target-language reasoning.

J Detailed Translation Quality and Translations Examples Results with Reasoning Models · K Discovered Problems in PolyMath Dataset · K.1 Incorrect Reference Answers

The translation assessment evaluates seven open-source reasoning models across FLORES+ and Sorbian test sets, finding generally poor chrF++ performance and model-dependent reasoning behavior. Annotation of PolyMath uncovered incorrect English reference answers and multilingual translation/localization errors, including three incorrect gold answers in the English split whose corrections were verified by annotators.

  • J Detailed Translation Quality and Translations Examples Results with Reasoning Models: Seven open-source models were tested on FLORES+ translation directions and German-to-Sorbian data, with Sorbian test sets sampled from 4,000 segments.The Sorbian samples contain 300 items, while other FLORES+ languages were evaluated both to and from English.
  • J Detailed Translation Quality and Translations Examples Results with Reasoning Models: chrF++ scores were generally poor across language pairs and models, while Olmo added the most information, producing long outputs and low scores.The evaluation used chrF++ because it supports all target languages.
  • J Detailed Translation Quality and Translations Examples Results with Reasoning Models: Reasoning length was more strongly determined by the model than the language pair and showed a weak negative correlation with chrF++; smaller local models often exhausted their thinking budget.Local models had a maximum thinking-token budget of 2048, with forced termination when the budget was reached.
  • J Detailed Translation Quality and Translations Examples Results with Reasoning Models: Translation quality, generation length, and empty-output rates were reported by model and direction in Tables 18–20, with higher chrF++ and lower empty-output rates preferred.Table 19 sums translation and reasoning tokens, while Table 20 reports answer and thinking emptiness separately.
  • K Discovered Problems in PolyMath Dataset: Annotators identified PolyMath errors in incorrect English reference answers and multilingual translation or localization, and the authors planned corrections as a pull request.The errors were found while annotating PolyMath and assessing its human reasoning traces.
  • K.1 Incorrect Reference Answers: low-en-94’s rabbit problem was corrected from 348 to 195, while high-en-122’s minimum-value problem was corrected from 729 to 328.Both corrections are reported alongside their original answers in the English split.
  • K.1 Incorrect Reference Answers: top-en-118 asks for the exact surface-area ratio of points within 2 · s of a cube vertex, but the supplied passage does not state its corrected answer.The problem defines distance along the cube surface and asks for area of D divided by total surface area.

K.2 Translation and Localization Errors · L LLMs Usage in This Work

The translated subsets contain recurring linguistic, formatting, factual, notation, and cross-lingual errors, including omissions that make some mathematical tasks ambiguous or unsolvable. Neural machine translation produced initial drafts, while humans handled quality assessment and refinement; LLMs also supported coding and language polishing, not the research itself.

  • K.2 Translation and Localization Errors: Translation issues were categorized as untranslated content, formatting errors, terminology problems, mistranslations, information loss, calques, notation errors, and cross-lingual misalignment.These labels capture both linguistic defects and structural inconsistencies across translated subsets.
  • K.2 Translation and Localization Errors: Some German items retain untranslated English, misuse formatting, or apply inconsistent terminology such as “Kisten” versus “Boxen.”Other German examples include missing commas and the unidiomatic phrase “3 Paar Shorts.”
  • K.2 Translation and Localization Errors: 35 Wochen pro Woche replaces the source’s 35 hours per week, creating a factually distorted German item.This is a mistranslation rather than a mere stylistic issue.
  • K.2 Translation and Localization Errors: low-de-110 is incoherent in both German and Czech and does not yield a solvable task.The defect affects both localized versions of the problem.
  • K.2 Translation and Localization Errors: Spanish errors include mistranslations, incorrect or context-unaware terminology, English calques, symbol-placement problems, and nonsensical translations.Examples include “secesión” for “sucesión,” “casco convexo” for “envoltura convexa,” and “se está poniendo demasiado pesada.”
  • K.2 Translation and Localization Errors: The release swaps Spanish and Vietnamese entries for high-es-46, additionally altering the Vietnamese LaTeX formatting, and predominantly uses Latin American Spanish.The mismatch is a cross-lingual release error, while the regional variety is a localization limitation.
  • L LLMs Usage in This Work: Neural machine translation generated only initial drafts; humans performed all subsequent quality assessment and refinement, while Claude assisted coding and ChatGPT edited language.The authors state that research ideas, experimental design, analyses, and findings were developed solely by them.
Loading 2607.05992v1…