PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens

Armel Zebaze, Rachel Bawden, Benoît Sagot

arXiv:2510.11919v1cs.CL

TL;DR

Whether natural-language intermediate tokens improve machine translation remains underexplored, so the paper evaluates thinking models and fine-tuning approaches across MT settings. It finds that thinking alone does not help, while traces containing translation attempts can improve fine-tuning; better target translations and more parallel data are more impactful than CoT distillation.

  • Problem

    The value of intermediate reasoning tokens for machine translation remains underexplored, despite their use in large reasoning models.

    Source: §1 · p. 2

  • Method

    The paper benchmarks thinking versus non-thinking models and fine-tunes students with human-inspired CoT or traces from modular MT prompting strategies.

    Source: §3 · p. 4, §4 · p. 5, §5.3 · p. 7

  • Results

    Thinking tokens do not improve MT, while modular-strategy traces can outperform input-output fine-tuning by up to 3.5 BLEU and 2 MetricX points.

    Source: §1 · p. 2, §5.3 · p. 7, §7 · p. 10

  • Takeaways & Limitations

    Translation attempts in traces matter more than explanations alone, while improving target translations or expanding parallel data brings greater benefits than CoT distillation.

    Source: §1 · p. 2, §5.3 · pp. 8–9, §7 · p. 10

Abstract

from arXiv · show

Large reasoning models (LRMs) have led to new possibilities in terms of problem-solving, through the devising of a natural language thought process prior to answering a query. While their capabilities are well known across mathematics and coding tasks, their impact on the task of machine translation (MT) remains underexplored. In this work, we explore the benefits of the generation of intermediate tokens when performing MT across multiple language pairs of different levels of resourcedness and multiple setups. We find that "thinking tokens" do not help LRMs better perform MT. This result generalizes to models fine-tuned to reason before translating using distilled chain of thought (CoT) inspired by human translators' practices. Specifically, fine-tuning a model with synthetic CoT explanations detailing how to translate step-by-step does not outperform standard input-output fine-tuning. However, constructing the intermediate tokens by combining the outputs of modular translation-specific prompting strategies results in improvements. Our findings underscore that the contribution of intermediate tokens during fine-tuning highly depends on the presence of translation attempts within them. More broadly, our results suggest that using a teacher to refine target translations or to expand parallel corpora is more impactful than distilling their CoT explanations into "thinking" MT models.

1 INTRODUCTION

The introduction asks whether natural-language intermediate tokens improve machine translation, a task where standard chain-of-thought prompting has not shown better results than vanilla prompting. It previews evidence that generic thinking and human-inspired CoT distillation do not help, while translation-strategy traces can.

1 INTRODUCTION

  • Across ten language directions and varied settings, LRM thinking does not significantly improve MT outputs.The experiments span zero- and few-shot settings, three benchmarks, high- and low-resource pairs, and different generation temperatures.Source: §1 · p. 2
  • Fine-tuning with synthetic human-inspired CoT explanations does not outperform standard input-output fine-tuning.The comparison uses gemma-3-4b-pt for English-to-Xhosa across six MT-specific CoT templates.Source: §1 · p. 2
  • Traces built from modular MT prompting strategies outperform standard input-output fine-tuning by up to 3.5 BLEU and 2 MetricX points.The reported gains stem mainly from translation attempts embedded in the traces.Source: §1 · p. 2
  • Using a teacher to improve target translations or expand parallel data has greater benefits than distilling CoT explanations into thinking MT models.The paper reports no additional inference cost after fine-tuning for these data-focused approaches.Source: §1 · p. 2

2 RELATED WORK

Related work traces chain-of-thought prompting from reasoning applications to thinking models, while noting doubts about trace faithfulness. In machine translation, standard CoT prompting has shown little success, motivating tests of distilled traces and modular translation strategies.

Reasoning with LLMs

  • Chain-of-thought prompting elicits natural-language reasoning before an answer and has been used in reasoning tasks and supervised fine-tuning data construction.Thinking models extend this approach by producing long chains of thought before responding.Source: §2 · p. 3
  • Prior studies question whether intermediate traces faithfully explain correct answers, and report that not thinking can outperform explicit reasoning on some tasks.The cited work also finds fine-tuning on incorrect traces can be as effective as on correct ones.Source: §2 · p. 3

Machine Translation with LLMs

  • LLMs perform well on high-resource MT but still struggle with low-resource languages, while few-shot examples and their selection affect translation quality.The literature also describes prompting strategies that iteratively refine translations, with or without external feedback.Source: §2 · p. 3
  • Standard CoT prompting has had little to no success in LLM-based MT, with most cited works reporting worse results than standard input-output prompting.Prior MT reasoning-model work has also used teacher-generated CoT for supervised fine-tuning followed by reinforcement learning.Source: §2 · p. 3
  • Modular MT strategies decompose translation into prompted steps whose outputs can be concatenated as intermediate tokens for student fine-tuning.Examples include source analysis, draft refinement, error annotation, self-feedback, and phrase decomposition.Source: §2 · p. 4

3 BENCHMARKING LRMS AT SCALE: TO THINK, OR NOT TO THINK?

The section compares translation with and without prior reasoning in Qwen3 models and examines generation temperature. Across tested language directions and temperatures, thinking offers little consistent advantage for MT.

3 BENCHMARKING LRMS AT SCALE: TO THINK, OR NOT TO THINK?

  • At temperatures 0.2–1.0, thinking is at most 1 BLEU point ahead but behind on MetricX, while lower temperatures correlate with higher MT performance.For Qwen3-32B, T = 0.0 works best in general; sampling degrades performance.Source: Figure 2 · p. 4, §3 · p. 4
  • Across ten English→X FLORES-200 directions, Qwen3 performance with and without prior thinking is similar, with non-thinking usually less than 0.5 MetricX point better.The comparison covers Qwen3 models from 0.6B to 32B parameters in zero-shot translation.Source: §3 · p. 4, Table 1 · p. 5

4 APPROACHES TO IMPROVING MT WITH INTERMEDIATE REASONING

The section tests whether teacher-generated intermediate information can help fine-tuned MT students, contrasting human-inspired CoT with traces from modular translation-specific strategies. In COTFT, the student generates intermediate tokens before the target translation.

4 APPROACHES TO IMPROVING MT WITH INTERMEDIATE REASONING

  • COTFT trains a student to generate intermediate tokens before the target translation, using teacher-generated information from parallel data.The approach is compared with input-output fine-tuning, where the student predicts the target directly from the source.Source: §4 · pp. 5–6
  • Human-inspired CoT distillation prompts a teacher to explain or justify how a given target translates the source, including linguistic analysis and context.The traces can identify sentence elements and reason about syntax and word order.Source: §4 · p. 5
  • The alternative uses concatenated outputs from modular MT prompting strategies as the student’s intermediate information.The teacher applies the selected strategy’s steps to the source sentence.Source: §4 · pp. 5–6
  • MAPS combines source analysis and translation attempts; SBYS uses research, drafting, refinement, and proofreading; TEaR drafts, annotates possible errors, and refines.These steps provide distinct intermediate outputs for the translation trace.Source: §4 · p. 6
  • Self-Refine iteratively improves a zero-shot draft, while CompTra decomposes the source, translates phrases with few-shot examples, and recombines them.Both strategies contribute their step outputs to the intermediate trace.Source: §4 · p. 6

5 EXPERIMENTS

The experiments compare standard input-output fine-tuning (IOFT) with chain-of-thought fine-tuning (COTFT) and translation-specific traces. CoT templates did not improve on IOFT, while translation attempts and better target translations could improve results.

5.1 EXPERIMENTAL SETUP

  • Experiments used FLORES-200 devtest (1,012 examples), with Xhosa and Lithuanian fine-tuning data and language-specific teacher–student model pairs.MetricX-24-Hybrid-XXL and BLEU were used for evaluation.Source: §5.1 · p. 6

5.2 DISTILLED CHAIN-OF-THOUGHT AS INTERMEDIATE TOKENS

  • COTFT fell short of IOFT by about 0.5 BLEU and 0.5 MetricX across all six human-translator-inspired CoT templates.Performance variability across templates was negligible; the same conclusion held with a different teacher.Source: §5.2 · p. 7, Figure 3 · p. 7

5.3 MT TRACES GENERATED BY PROMPTING STRATEGIES AS INTERMEDIATE TOKENS

  • 18 BLEU and 5.6 MetricX with IOFT-BOA, versus 14 BLEU and 8 MetricX, using the same parallel pairs and training recipe.IOFT with better ground truths consistently outperformed trace-based curves and was cheaper and faster to train.Source: Figure 5 · p. 8, §5.3 · pp. 8–9
  • For MAPS, SBYS, TEaR, and Self-Refine, IOFT-MAX generally outperformed COTFT-MAX when traces lacked attempts better than the ground truth.COTFT-MAX with CompTra did slightly better than IOFT-MAX, but both underperformed COTFT with CompTra.Source: §5.3 · p. 8, Figure 6 · p. 9

6 DISCUSSION AND ANALYSIS

Sentence-decomposition traces and parallel-data augmentation improved some fine-tuning comparisons, while GRPO did not make COTFT outperform IOFT. Short, overlapping generated pairs helped less as added training data than novel sentences.

6.1 DOWN THE RABBIT HOLE OF SENTENCE DECOMPOSITION

  • IOFT-EXT(P) and IOFT-EXT(SP) gained +4 BLEU and −2 MetricX over IOFT, and were the best overall approaches.They also outperformed COTFT with P and SP; H and CompTra augmentation was less successful.Source: Figure 7 · p. 9, §6.1 · pp. 9–10
  • COTFT beat IOFT across the four decomposition strategies, but short H and CompTra pairs yielded fewer augmentation gains because they overlapped substantially with original samples.Those short phrase-level pairs remained useful as COTFT intermediate information.Source: §6.1 · pp. 9–10

6.2 REINFORCEMENT LEARNING AFTER IOFT AND COTFT

  • After GRPO, COTFT still did not outperform IOFT; continuing IOFT fine-tuning gained +6 BLEU and −3 MetricX, versus about +1.3 BLEU and −1.0 MetricX across the GRPO setups.The post-SFT ordering remained unchanged after RL.Source: §6.2 · p. 10

7 CONCLUSION

The paper concludes that intermediate traces help when they contain translation attempts, but not when such attempts are absent. It identifies target-translation quality and parallel-data quantity as key factors in MT fine-tuning.

7 CONCLUSION

  • Intermediate tokens provided no benefit without full or partial translation attempts, but accounted for all improvements when translation attempts were present.The conclusion states that parallel data is crucial for improving MT both with and without intermediate tokens.Source: §7 · p. 10

A.1 MODELS, DATASETS AND TOOLS

The appendix table lists links to experimental resources.

A.1 MODELS, DATASETS AND TOOLS

  • Table 2 provides links to the datasets, benchmarks, and models used in the experiments.The passage does not specify the individual resource links.Source: Table 2 · p. 28, §A.1 · p. 28

A.2 IMPLEMENTATION DETAILS

The implementation uses target-only loss and bootstrap resampling for significance testing. It constructs teacher-generated CoT traces from translation templates and modular prompting strategies.

A.2 IMPLEMENTATION DETAILS

  • Training loss is computed only on the translation target or intermediate tokens followed by the translation, with the same prompt used for evaluation.During COTFT, the target includes intermediate tokens inside <think> tags followed by the final translation.Source: §A.2 · p. 28
  • Significance is assessed with bootstrap resampling using 300 samples of 500 sentences and a 0.05 p-value threshold.Source: §A.2 · p. 29

A.3 COT CONSTRUCTION TEMPLATES

  • Six CoT templates model human translation strategies, and teachers generate first-person explanations of how a source maps to its provided translation.The explanations are produced using a prompt tied to each template and can later be used for COTFT.Source: §A.3 · pp. 29–30

A.4 PROMPTING STRATEGIES

  • Translation-specific strategies generate candidate drafts from extracted information or progressively refine translations through research, assessment, proofreading, and selection.Examples include MAPS, which combines keywords, topics, demonstrations, and a zero-shot attempt, and SBYS, which researches challenging expressions before drafting and refinement.Source: §A.4 · pp. 31–33

A.5 RL TRAINING HYPERPARAMETERS

GRPO training uses four H100 GPUs, 5,000 steps, and LoRA adapters on selected attention and MLP projection modules.

A.5 RL TRAINING HYPERPARAMETERS

  • GRPO training runs for 5,000 steps, including 100 warmup steps, on four H100 GPUs, with one GPU dedicated to reward computation.Source: §A.5 · p. 33
  • Each device uses batch size 4 with four gradient-accumulation steps; generation samples 12 outputs per prompt for an effective batch size of 48.Source: §A.5 · p. 33
  • LoRA fine-tunes seven projection modules with rank 32, scaling factor 64, and dropout 0.05.The modules are q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj.Source: §A.5 · p. 33

B.1 BENCHMARKING LRMS AT SCALE: TO THINK, OR NOT TO THINK?

Across tested LRM sizes and translation settings, generating thinking tokens yields at most small, inconsistent gains; for models large enough to avoid wrong-language outputs, it brings no gains. Few-shot demonstrations do not change that pattern.

B.1 BENCHMARKING LRMS AT SCALE: TO THINK, OR NOT TO THINK?

  • Thinking tokens yield only small, inconsistent gains across the reported chrF++ and XCOMET comparisons, while larger models show no gains once wrong-language outputs are resolved.For smaller models, thinking can help when they otherwise answer in English or Chinese instead of the target language.Source: §B.1 · p. 33, Table 3 · p. 34
  • The additional DeepSeek-R1-Distill model comparisons show the same pattern: “no thinking” consistently outperforms thinking.Source: §B.1 · p. 33, Table 4 · p. 34
  • Providing five-shot demonstrations does not lift thinking-mode performance, either for English-to-X or X-to-English translation.Source: §B.1 · p. 34

B.2 RESULTS ON NTREX-128 AND TICO-19

Across the additional benchmarks and fine-tuning variants, ordinary reasoning traces do not consistently improve translation, while translation attempts and better targets can help. GRPO improves scores across setups but does not favor COTFT over IOFT.

B.2 RESULTS ON NTREX-128 AND TICO-19

  • On NTREX-128, thinking matches the FLORES-200 pattern, while on TICO-19 it offers little to no advantage, including for some low-resource languages.NTREX-128 evaluation uses the first 1,000 of its 1,997 pairs; TICO-19 covers 35 languages and has a 2,100-sample test set.Source: §B.2 · pp. 34–35

B.3 DOES DISTILLED CHAIN-OF-THOUGHT AS INTERMEDIATE TOKENS IMPROVE PERFORMANCE?

  • COTFT with all six distilled CoT templates lags IOFT in English-to-Lithuanian; even the best template, T3, remains behind.The gap reaches 5 BLEU and 4 MetricX.Source: §B.3 · p. 39, Figure 8 · p. 39

B.4 WHAT HAPPENS WHEN WE USE TRACES FROM MT PROMPTING STRATEGIES AS INTERMEDIATE TOKENS?

  • Prompting-strategy traces usually let COTFT outperform IOFT, but SBYS is an exception; higher-quality IOFT-BOA targets beat standard IOFT by 3 BLEU and 1.3 MetricX.COTFT-MAX does not improve over IOFT-MAX, and IOFT-BOA tops both with the same number of parallel pairs and training recipe.Source: §B.4 · p. 39, Figure 9 · p. 39, Figure 10 · p. 40

B.5 DOWN THE RABBIT HOLE OF SENTENCE DECOMPOSITION

  • COTFT outperforms IOFT across decomposition strategies; COTFT with CompTra matches IOFT-EXT(P), which requires six times as many parallel pairs.Adding generated pairs is especially helpful for P and SP, while short phrases and their translations are useful intermediate information for COTFT.Source: §B.5 · p. 40, Figure 11 · p. 40

B.6 COT DISTILLATION: WHAT HAPPENS WHEN WE CHANGE THE TEACHER?

  • With DeepSeek-R1-Distill-Llama-70B as teacher, COTFT behaves similarly to IOFT across templates but does not improve over it.IOFT is also faster to train.Source: §B.6 · p. 40, Figure 12 · p. 41

B.7 MT TRACES GENERATED BY PROMPTING STRATEGIES AS INTERMEDIATE TOKENS: WHAT HAPPENS WHEN WE CHANGE THE TEACHER?

  • Changing the teacher to Gemma-3-27B makes COTFT outperform IOFT except with SBYS, while Gemma-3-4B traces fail to help COTFT exceed IOFT.With the larger teacher, IOFT-BOA works best; with the smaller teacher, performance degrades, consistent with traces helping only when their translation attempts exceed the ground truth in quality.Source: §B.7 · p. 41, Figure 13 · p. 41

B.8 REINFORCEMENT LEARNING AFTER IOFT AND COFT

  • GRPO adds approximately +1 BLEU and -0.7 MetricX across setups but preserves IOFT/COTFT ordering; IOFT alone can match or exceed those gains at lower computational cost.Source: §B.8 · pp. 41–42, Figure 15 · p. 42

Related papers

Loading 2510.11919v1…