PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual Environments

Patomporn Payoungkhamdee, Pume Tuchinda, Jinheon Baek, Samuel Cahyawijaya, Can Udomcharoenchaikit, Potsawee Manakul, Peerat Limkonchotiwat, Ekapol Chuangsuwanich, Sarana Nutanong

arXiv:2502.17956v2cs.CL

TL;DR

PoT multilingual reasoning remains underexplored, despite persistent challenges in non-English reasoning. The paper evaluates PoT fine-tuning and reasoning quality, finding improved alignment over CoT fine-tuning and a strong link between reasoning quality and answer accuracy.

  • Problem

    PoT multilingual reasoning remains underexplored compared with non-English CoT fine-tuning, motivating research into its alignment and generalization across languages.

    Source: §1.1 · p. 1

  • Method

    The study separates multilingual reasoning from code execution to examine question-reasoning alignment, reasoning quality, answer correctness, and ICE-Score-guided Soft-SC test-time scaling.

    Source: §1.2 · p. 2, §6 · p. 8

  • Results

    PoT fine-tuning substantially enhances multilingual reasoning alignment and generalization, outperforming CoT fine-tuning; reasoning quality also strongly correlates with answer accuracy.

    Source: §6 · p. 8

  • Takeaways & Limitations

    ICE-Score-based inference strategies improve performance, particularly in low-resource languages, supporting code quality as a test-time scaling heuristic.

    Source: §1.2 · p. 2, §6 · p. 8

    The evaluation uses GSM8K grade-school math problems, so its findings may not generalize to other reasoning-intensive tasks.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Multi-step reasoning is essential for large language models (LLMs), yet multilingual performance remains challenging. While Chain-of-Thought (CoT) prompting improves reasoning, it struggles with non-English languages due to the entanglement of reasoning and execution. Program-of-Thought (PoT) prompting separates reasoning from execution, offering a promising alternative but shifting the challenge to generating programs from non-English questions. We propose a framework to evaluate PoT by separating multilingual reasoning from code execution to examine (i) the impact of fine-tuning on question-reasoning alignment and (ii) how reasoning quality affects answer correctness. Our findings demonstrate that PoT fine-tuning substantially enhances multilingual reasoning, outperforming CoT fine-tuned models. We further demonstrate a strong correlation between reasoning quality (measured through code quality) and answer accuracy, highlighting its potential as a test-time performance improvement heuristic.

1 Introduction

LLMs’ weaker performance in non-English languages and the limited study of multilingual PoT motivate an evaluation of question-to-reasoning alignment and reasoning-to-answer quality. The paper reports that fine-tuning and code-quality-guided inference can improve multilingual PoT performance.

1 Introduction

  • Multi-step reasoning supports complex tasks, but LLMs perform worse in non-English languages amid linguistic and training-data differences.The introduction also frames multilingual reasoning as a way to expand access to these capabilities.Source: §1 · p. 1

1.1 Research Gap

  • Multilingual PoT remains underexplored relative to non-English CoT, with only one cross-lingual prompting study reported (Ranaldi et al., 2024a).The paper identifies PoT fine-tuning as a route to investigate generalization to unseen languages and multilingual performance.Source: §1.1 · p. 1

1.2 Problem Formulation

  • The framework separates generating reasoning steps from questions (Q→R) and executing them to obtain answers (R→A), examining fine-tuning for alignment and reasoning quality’s relation to correctness.It compares English-only fine-tuning with cross-lingual evaluation against training that includes target-language samples.Source: §1.2 · pp. 1–2
  • Removing inline comments improves generalization to unseen languages in cross-lingual fine-tuning, while translating comments into the target language improves alignment and overall performance in multilingual fine-tuning.For CodeLlama-7B, comments help English and Bengali, but removing them improves German and the overall score.Source: §1.2 · p. 2
  • 31.6% to 56.6% overall accuracy in cross-lingual settings with ICE-Score-based Soft Self-Consistency, outperforming standard Self-Consistency.The method uses code quality as a test-time soft-voting heuristic.Source: §1.2 · p. 2

1.3 Contributions

  • The paper contributes a framework for multilingual PoT, a systematic evaluation of fine-tuning and inline comments, and an analysis of code quality’s relationship to answer accuracy.It also uses that relationship to improve test-time inference.Source: §1.3 · p. 2

2 Proposed Studies

The proposed studies examine multilingual PoT through question-to-reasoning alignment and the relationship between code quality and answer correctness. They vary question and comment languages, assess code quality, and explore its use in test-time selection.

2.1 Fine-tuning for Q-R Alignment (P1)

  • CoT-guided few-shot prompting produced PoT samples with 96.1% correctness, supporting construction of the GSM8KPoT dataset.The study varies prompting strategies to generate program solutions from GSM8K problems.Source: §2.1 · p. 2

2.1.1 Cross-lingual Setup

  • Cross-lingual fine-tuning compares English-comment GSM8KPoT with a version whose inline comments are removed.The study varies question and comment languages to assess fine-tuning strategies; NC means no comment.Source: §2.1.1 · p. 3, Table 1 · p. 3, §2.1 · p. 3

2.1.2 Multilingual Setup

  • The multilingual setup includes four datasets that vary whether questions and comments are cross-language, parallel, absent, or translated into target languages.Variants include translated comments and translated questions paired with English comments.Source: §2.1.2 · p. 3

2.2 Code Quality Analysis (P2)

  • Code quality is assessed with ICE-Score, and the study tests its association with answer accuracy at both system and sample levels.Functional correctness is rated from 0 (incorrect or incomplete) to 4 (fully correct); system-level Spearman correlation and sample-level AUC and t-tests are used.Source: §2.2 · p. 4
  • The authors report a strong correlation between code quality and answer accuracy and use ICE-Score in Soft Self-Consistency to rank answer candidates.Soft voting by code quality outperforms standard Self-Consistency with hard voting and improves overall accuracy across languages in cross-lingual settings.Source: §2.2 · p. 4

2.3 Discussions

  • The discussion frames language availability and inline comments as factors in PoT reasoning, with code quality serving as an intermediate link to final-answer accuracy.The study asks how multilingual data availability affects reasoning steps and how comments affect performance across language setups.Source: §2.3 · p. 4

3 Experimental Setup

The experiments compare model variants and fine-tuning data for multilingual reasoning, then evaluate zero-shot accuracy on MGSM across ten languages.

  • The model set includes Llama2-7B, CodeLlama-7B, Llama2-13B, and Llama3 8B; Llama3.1-405B Instruct serves as the Oracle for dataset generation and code assessment.The variants cover code-specific, model-size, and model-version differences.Source: §3 · p. 4
  • Accuracy is evaluated zero-shot on MGSM across English, German, French, Spanish, Russian, Chinese, Japanese, Thai, Swahili, and Bengali.Greedy decoding is used; PoT programs are executed in Python and their outputs compared with labels.Source: §3 · p. 4
  • Fine-tuning compares GSM8K and GSM8KPoT for cross-lingual models and parallel or cross-language MGSM8K-Instruct data for multilingual CoT and PoT.The setups also include variants with comments removed and machine-translated inline comments.Source: §3 · pp. 4–5

4 Experimental Results

Across cross-lingual and multilingual settings, PoT generally outperforms CoT, while fine-tuning choices affect question–reasoning alignment. Code quality correlates with answer accuracy and can guide test-time scaling.

4.1.1 Cross-lingual Setting

  • PoT outperforms CoT in 39 of 40 cross-lingual cases, with Swahili on Llama2-13B the sole exception.In that exception, PoT scores 7.2% and CoT 7.4%.Source: §4.1.1 · p. 5
  • Removing comments tends to improve non-English accuracy for Llama2 models, while CodeLlama-7B shows mixed language-specific results.For CodeLlama-7B, comments help in English and Bengali, whereas removing them improves German and the overall score; Llama3-8B improves across all languages when comments are removed.Source: §4.1.1 · p. 5

4.1.2 Multilingual Setting

  • PoT retains an overall advantage over CoT in multilingual settings, and translating inline comments into the target language yields the best alignment strategy across model variants.Llama3-8B reaches 76.5% accuracy in English, 57.6% in Thai, and 55.2% in Bengali.Source: §4.1.2 · p. 6

Does Better Strategy Improve Code Quality?

  • Higher ICE-Score code quality correlates with higher accuracy, and ICE-Score-guided Soft-SC improves Llama2-7B accuracy from 39.2 to 56.6 cross-lingually and from 57.2 to 71.2 multilingually.The system-level Spearman correlations are 0.91 cross-lingually and 0.76 multilingually; ICE-Score also distinguishes correct from incorrect answers with AUC values of 0.94 and 0.96, respectively.Source: Does Better Strategy Improve Code… · p. 7

5 Related Work

Prior work improves mathematical reasoning through intermediate reasoning, instruction fine-tuning, and tool use, while multilingual approaches include translation, preference optimization, and cross-language prompting.

Mathematical Reasoning

  • CoT uses step-by-step reasoning, while PoT and PaL delegate computation to a Python interpreter; self-consistency samples multiple solutions and selects the most consistent.Instruction fine-tuning approaches include MetaMathQA’s diverse rephrasings and MathInstruct’s combination of CoT generality and PoT computational precision; curated tool-use datasets are another approach.Source: §5 · p. 8

Multilingual Mathematical Reasoning

  • Multilingual mathematical reasoning methods include translated datasets, translating questions into English before fine-tuning, and multilingual preference optimization.The passage notes that accurate dataset translation can be costly and time-consuming.Source: §5 · p. 8
  • Prompting approaches include translating questions before CoT, multilingual Tree-of-Thought, and Cross-PAL, which aligns reasoning across languages through code generation.Source: §5 · p. 8

6 Conclusion

The paper studies multilingual PoT by separating reasoning from execution and evaluating fine-tuning and reasoning quality. It reports improved alignment and generalization over CoT fine-tuning, alongside a strong association between code quality and answer accuracy.

  • PoT fine-tuning enhances reasoning alignment and generalization across cross-lingual and multilingual settings, outperforming CoT fine-tuning.Source: §6 · p. 8
  • The paper reports a strong correlation between reasoning quality and answer accuracy, and says ICE-Score-based inference strategies enhance performance, particularly in low-resource languages.The authors identify extending the framework to other reasoning-intensive tasks and exploring more advanced alignment techniques as future work.Source: §6 · p. 8

Limitations

The study’s findings are bounded by GSM8K’s grade-school math scope and the ICE-Score model used for test-time scaling. Its MGSM8KInstruct reference data distinguishes cross and parallel CoT training strategies.

Limitations

  • GSM8K results may not generalize to other reasoning-intensive tasks, and potential benchmark contamination has been raised.The authors retain GSM8K for comparability and plan to extend evaluations to additional benchmarks.Source: Limitations · p. 9
  • The magnitude of Soft-SC gains may depend on the ICE-Score model, so other configurations and their cost-benefit trade-offs remain to be examined.The study uses Llama3 405B for ICE-Score calculations in both correlation and test-time scaling experiments.Source: Limitations · p. 9

A MGSM8KInstruct

  • MGSM8KInstruct pairs English reasoning with English questions and translations of those questions into nine additional languages.It is used as the reference dataset for CoT in multilingual settings.Source: A MGSM8KInstruct · p. 11
  • CoT Cross uses English questions with target-language answers, whereas CoT Parallel uses question-answer pairs in the same language.The strategies are presented as ways to promote multilingual adaptability and strengthen capability within target languages, respectively.Source: A MGSM8KInstruct · p. 11

B PoT Generation Methods

The authors compare zero-shot, few-shot, and CoT-guided few-shot PoT synthesis, finding the CoT-guided approach yields the highest correctness rate. The prompts direct an Oracle LLM to generate Python solutions for GSM8K problems.

B PoT Generation Methods

  • Few-shot PoT adds two solved examples, and CoT-guided few-shot adds their reasoning alongside the corresponding Python solutions.The Oracle LLM used for the comparison was Llama3.1 405B Instruct.Source: B PoT Generation Methods · p. 11
  • 96.1% correctness for few-shot PoT with CoT guidance exceeded few-shot PoT at 94.5% and zero-shot PoT at 58.7%.The metric is correctness of the synthesized PoT samples.Source: B PoT Generation Methods · p. 12
  • The zero-shot prompt template uses [Question] as a placeholder for the problem statement.The model is prompted to generate a solver() function in Python without exemplars.Source: B PoT Generation Methods · p. 11, Figure 4 · p. 12

C Training Setting

The section specifies fine-tuning and computing settings, reports test-time scaling comparisons, and describes Llama3-8B’s sensitivity to prompt and hyperparameter choices.

C Training Setting

  • Fine-tuning used three epochs and a maximum sequence length of 1024, with different learning rates and batch sizes for Llama3 8B than for the other listed models.Llama3 8B used a 5e-6 learning rate and effective batch size 128; Llama2-family models and CodeLlama used 2e-5 and 512.Source: C Training Setting · p. 12

E Comparison with Non-Fine-Tuned PoT

D Computing Resources

  • In summary, the experiments required 544 A100 GPU-hours for fine-tuning, 52 hours for inference, and 146 hours for quality assessment, as reported for the study overall.Fine-tuning used 4× NVIDIA A100 (80GB) GPUs; a separate dedicated 4× NVIDIA A100 (80GB) setup hosted Oracle LLM inference for constructing PoT answers and evaluating code quality.Source: D Computing Resources · p. 12

F Sensitivity of Llama3-8B

  • Llama3-8B’s non-compilable output rate fell from 9.12% to 1.68% after hyperparameter and chat-template changes and adding a def solver(): prefix.The initial rate was higher than the reported rates for Llama2-7B (3.08%), CodeLlama-7B (2.04%), and Llama2-13B (1.84%).Source: F Sensitivity of Llama3-8B · p. 13, Figure 7 · p. 13

G Alternative Metric For Code Quality Assessment

The section reports a limitation of CodeBERT-Score for distinguishing small arithmetic errors in GSM8K-style code and summarizes the main fine-tuning findings across language settings.

G Alternative Metric For Code Quality Assessment

  • CodeBERT-Score varied only ±1.0% across systems and failed to reflect accuracy differences, limiting its suitability for GSM8K-style correctness evaluation.The authors attribute this to short code snippets with small numerical or arithmetic mistakes that semantic similarity may not distinguish.Source: G Alternative Metric For Code Quality… · p. 14

H.1 Main Results

  • PoT fine-tuning improves multilingual reasoning; cross-lingual generalization benefits from removing inline comments, while multilingual alignment benefits from translating them into target languages.The full results are reported in Tables 10 and 11.Source: §H.1 · p. 14

H.2 Code Analysis

The full code-quality results are consistent with earlier findings, showing a strong correlation between reasoning quality and final answer correctness. The observed trends support using code quality for test-time scaling, with particularly notable improvements in underrepresented languages.

H.2 Code Analysis

  • The full code-quality results confirm a strong correlation between reasoning quality and final answer correctness.The analysis reports consistency with findings in Section 4.2.Source: §H.2 · p. 14
  • Code-quality evaluation reports ICE-Score and CodeBERT-Score results for cross-lingual and multilingual PoT variants on MGSM.Tables 12–13 cover ICE-Score; Tables 14–15 cover CodeBERT-Score (F1).Source: §H.2 · p. 14, Table 12 · p. 15, Table 13 · p. 16
  • The observed trends support leveraging code quality for test-time scaling, with improvements in underrepresented languages particularly notable.Source: §H.2 · p. 14

Related papers

Loading 2502.17956v2…