Source-linked AI summary
Ladders in Chaos: When, How, (and Perhaps Why) Does Test-Time Scaling Improve LLM Machine Translation
Di Wu, Sergey Troshin, Christof Monz, Antske Fokkens, Vlad Niculae
TL;DR
The paper asks how sequential and parallel test-time scaling differ in machine translation and studies them in a simplified, controlled framework. It finds sequential sampling more sample-effective, with gains partly linked to larger target-side context, but accuracy can decline at larger budgets and results depend on context construction and evaluation scope.
Problem
The mechanisms and task-dependent effectiveness of test-time scaling in machine translation remain insufficiently understood.
Method
The study compares independently sampled and reranked translations with sequential self-refinement under controlled sampling budgets, manual evaluation, and context ablations.
Results
Sequential sampling explores output spaces more effectively, improves fluency and naturalness, can lose accuracy at larger budgets, and gains partly reflect larger target-side context.
Takeaways & Limitations
Sequential test-time scaling is a strong but context-sensitive translation strategy whose benefits are greatest under limited budgets.
Takeaways & Limitations
The experiments use a single dataset, a small subset of languages, and a single prompt format, with only one sequential trajectory per run.
Abstract
from arXiv · showhide
Two forms of test-time scaling for Large Language Models (LLMs) have emerged as effective and widely adopted paradigms: sequential, in which later answer attempts depend on earlier ones, and parallel, such as i.i.d. sampling with reranking. In this study, we investigate their properties in translation. First, our study shows that sequential sampling has a higher performance ceiling, providing a more diverse and effective pool of samples, particularly under smaller sampling budgets. Second, we interrogate the nature of test-time scaling through a multidimensional manual analysis. Human analysis of the Best-of-$N$ translations demonstrates that sequential sampling substantially improves translation fluency and naturalness, but can degrade accuracy when inference budgets are large. Finally, we suggest an explanation of the mechanism through which sequential scaling improves machine translation. Our controlled analysis partially attributes the success of sequential self-improvement to the model's access to a larger target-side context. Ablation experiments on sequential sampling demonstrate its robustness across different sampling temperatures, while also revealing sensitivity to context construction, suggesting directions for future improvement.
1 Introduction
The paper compares parallel and sequential test-time scaling for machine translation in a controlled setting, finding that sequential sampling is more sample-effective but has context- and budget-dependent trade-offs.
- The study compares independent parallel sampling with reranking against sequential self-improvement across controlled sampling budgets.
- Sequential sampling explores output spaces more effectively than parallel sampling, especially under limited budgets.
- Human evaluation finds that sequential sampling improves translation fluency, accuracy, and naturalness, but can lose accuracy relative to parallel sampling at larger budgets.
- Controlled experiments partially attribute sequential gains to access to larger target-side context rather than solely to self-reflection.
- Restricting sequential context to a single generation round produces clear gains across metrics, showing that additional context can be counterproductive.
2 Background
The background motivates studying simple iterative self-refinement because test-time scaling mechanisms remain unclear and their effectiveness varies across tasks and approaches.
- Test-time scaling adds inference-time computation, commonly through chain-of-thought and iterative multi-turn generation.
- The mechanisms behind test-time scaling remain unclear, and its effectiveness varies considerably across domains.
- In translation, explicit chain-of-thought decomposition lacks clear benefits, whereas prompting models to translate again yields better results.
- The study uses a minimal self-refinement setting and an external metric only for Best-of-N selection, never to guide generation.
- Sequential sampling is studied alongside parallel sampling, which supports quality-aware or minimum-Bayes-risk decoding, and related sequential methods for sampling efficiency.
3 Sequential and Parallel Sampling
The paper formalizes parallel and sequential translation sampling under matched generation budgets, with selector and non-selector variants for comparing performance and diversity.
- Both parallel and sequential algorithms are applied up to a fixed number of generation rounds to ensure fair budget comparisons.
- Parallel sampling independently draws multiple translations conditioned on the same source prompt.
- Sequential sampling generates translations across multiple turns, using previous translations as context for subsequent generation.
- The sequential process repeatedly asks the model to revisit a translation and improve it using the same refinement prompt.
- The study evaluates parallel-with-selector, sequential-without-selector, and sequential-with-selector settings across varying sampling budgets.
4 Experimental Setup
Experiments use WMT24++ and several language-model and metric configurations to evaluate sampling strategies across six supported translation directions.
- WMT24++ is used because it covers varied domains and contains human-written references post-edited by professional translators.
- Experiments cover six high- and medium-resource directions from English into Chinese, German, Russian, Dutch, Romanian, and Arabic.
- The models include Qwen3-32B, Qwen3-4B, and the commercial GPT-4o-mini.
- CometKiwi-XL is the default reference-free selector, while MetricX24, COMET, and ReMedy-9B-22 provide additional quality measures.
- Main results report the selected sample or final self-refinement output at each budget, where budget counts translation rounds.
5 An Overview of Sampling Effectiveness
Sequential sampling is especially effective at low computation budgets, while selectors improve both sequential and parallel sampling as budgets grow. Early “Translate Again” rounds yield strong gains, but later rounds can plateau or degrade quality.
- Selector effects: Adding an external selector substantially improves both parallel and sequential sampling, including at very limited budgets such as two samples.Performance gains continue as sampling size increases and begin to converge around budget-30.
- Sampling effectiveness: Sequential sampling becomes less effective in later rounds as marginal diversity declines, producing a plateau and, for COMET, possible quality degradation.The later-round degradation is associated with reduced marginal diversity and repetition.
- Selector effects: With a selector, sequential sampling can select better samples at higher budgets, indicating that the language model alone does not consistently improve its samples over additional rounds.The comparison motivates treating selector-based sequential sampling separately from sequential sampling without a selector.
- Sampling effectiveness: Sequential sampling outperforms parallel sampling at smaller computational budgets across all metrics, though parallel can win on COMET at higher budgets.For MetricX-24, sequential sampling consistently outperforms parallel sampling; the pattern also holds for commercial LLMs and the smaller Qwen3 model.
6 Analysis
Human evaluation finds that sequential translation improves fluency and style relative to parallel sampling but can lose accuracy at larger budgets. Controlled experiments link part of sequential self-improvement to expanded target-side context and reduced left-to-right asymmetry.
- 6.1 Human evaluation: Human evaluation compares sequential and parallel translations along MQM accuracy, fluency, and style dimensions using unweighted preference and severity-split analyses.The comparisons include sequential turn-1 versus turn-2, selector-based sequential versus turn-1, and selector-based sequential versus parallel.
- 6.1 Human evaluation: Sequential sampling improves all three MQM dimensions over direct translation after one additional round, with larger gains when combined with a selector at higher budgets.The severity analysis also identifies several critical improvements attributed mainly to sequential sampling.
- 6.1 Human evaluation: At budget-30, sequential samples are more fluent and stylistically preferable, whereas parallel samples are more accurate.The authors suggest that weakening source-sentence influence during multi-turn generation may reduce sequential faithfulness.
- 6.2 On the Success of “Translate Again”: The controlled experiment slices WMT24++ documents into two consecutive-sentence windows, applies “Translate Again,” and compares translation quality at HEAD and TAIL positions.Sentences are arranged so each appears exactly once in each position, with T1 and T5 discarded for strict comparability.
- 6.2 On the Success of “Translate Again”: Autoregressive left-to-right generation gives later segments access to preceding translated outputs but denies earlier segments their right context.A second-pass translation can access the full global context generated in the first pass, providing a concrete mechanism for part of the observed improvement.
- 6.2 On the Success of “Translate Again”: First-pass HEAD translations underperform TAIL translations, while a second pass recovers HEAD performance to a comparable level.This pattern supports the role of asymmetric context in sequential self-refinement.
- 6.3 Diversity analysis: Marginal diversity analysis defines novel samples through exact-match distance and shows that sequential sampling initially produces far fewer duplicates than parallel sampling.Sequential duplication increases in later rounds, where repetition bias appears and helps explain the plateau effect.
7 Ablation study
Ablations show that sequential sampling remains useful across model sizes and temperatures, but its performance and diversity depend strongly on how much prior output is retained in context.
- Model size: The smaller Qwen3-4B model shows the same pattern of better sequential performance at smaller budgets, but with a less pronounced advantage over parallel sampling.This may indicate weaker self-refinement ability in smaller models.
- Context ablation: Constrained sequential context improves almost every metric and language direction, while smaller context windows reduce COMET degradation in self-refinement.The context-controlled setting also reduces repetition bias, improving sampling effectiveness while lowering context costs.
- Evaluation setup: Table 1 aggregates six-language automatic results for sequential self-refinement and budget-30 Best-of-N comparisons across context sizes and parallel sampling.The table contrasts turn-1, turn-2, and turn-3 sequential refinement with full-context, h = 5, h = 1, and selector-based parallel settings.
- Context ablation: Figure 6 summarizes that smaller sliding-window contexts reduce COMET degradation and context control reduces diversity collapse.The ablation also tests sequential sampling under low temperatures.
- Temperature ablation: Sequential sampling remains useful in the low-temperature limit, including greedy decoding, whereas parallel sampling eventually loses diversity as temperature approaches greedy decoding.This suggests a quality–diversity trade-off for future sampling methods.
8 Conclusion
Across open- and closed-source models, the study finds sequential test-time scaling superior in sample efficiency and duplicate avoidance. The analyses attribute part of this advantage to context processing and show that context control stabilizes sequential sampling.
- Conclusion: Experiments on open-source and closed-source models compare sequential and parallel test-time scaling paradigms.
- Conclusion: Sequential sampling is superior in sample efficiency and low output-space duplicate rate.
- Conclusion: Context processing partially explains sequential sampling’s success, while context-constrained settings produce more stable sequential sampling.The conclusion presents context control as a direction for further improvement.
9 Limitations
The study’s limitations include restricted prompting, model and dataset coverage, and incomplete analysis of diversity and context construction. It also warns that fluent outputs may conceal inaccuracies and that selector or model biases can shape results.
- Prompting and generalization: The experiments use a single prompt format, so findings may change with different refinement instructions.The authors specifically mention purely diversity-focused alternatives.
- Prompting and generalization: Sequential reflection abilities vary across models, limiting how confidently the findings generalize to other models.Generalization depends on the capabilities of the model being tested.
- Practical risks: Fluent and natural sequential translations may contain minor inaccuracies that are difficult for users to detect in practical MT settings.The authors identify fluent but unfaithful translations as a practical risk.
- Data and evaluation scope: The evaluation uses one dataset and a small subset of languages, excluding low-resource and extremely low-resource settings.The MQM analysis covers only three translation directions and one sequential trajectory per run, limiting quality-score stability.
- Analysis scope: The analysis of mechanisms is limited to hard duplicate matching and a single sliding-window strategy for compacting context.The authors suggest soft-matching utility functions and other context-compaction strategies as extensions.
10 Ethical considerations
The study acknowledges that generation settings and selector metrics can favor or suppress particular translation attributes. Its experiments use controlled prompts, fixed computational settings, and human evaluation procedures designed for comparability.
- Bias and selection: Best-of-N results inherit biases from the selector metric, which determines which translations are selected.Even an unbiased selector may interact with model biases under limited context.
- Generation settings: Main experiments use temperature 1.0 to compare parallel and sequential sampling in a high-diversity regime.Lower temperatures improve selector-free sequential sampling by favoring precision over diversity.
- Human evaluation: Human evaluators are native target-language speakers who compare shuffled translation pairs on accuracy, fluency, and style.Shuffling is used in the blind comparison setup to mitigate position bias.
- Data and evaluation: The dataset contains 633 two-sentence segments constructed from 171 documents and 998 sentences in WMT24++.A special separator connects the two sentences before sentence-level scoring.
F Results
The results section reports experiments across Qwen3-32B, Qwen3-4B-Instruct-2507, and GPT-4o-mini, comparing sampling strategies and context-window variants. The main Qwen3-32B results include sequential turns and Best-of-N configurations.
- Experimental results: Main Qwen3-32B results compare six language pairs across sequential and parallel sampling strategies.Table 1 also includes sequential selector results with full, context-5, and context-1 windows.
- Experimental results: Additional experiments evaluate Qwen3-4B-Instruct-2507 using the same quality-versus-budget comparison.The results are presented in Figure 11.
- Experimental results: GPT-4o-mini experiments report full-context sequential results alongside context-5 and context-1 ablations.These configurations are presented in Figures 12–14.
G Human Evaluation Results
Human evaluation finds that sequential sampling improves several translation qualities, while parallel sampling often retains an accuracy advantage at larger budgets. Context management and selector choice further shape the observed scaling behavior.
- Human evaluation: Sequential sampling improves fluency, accuracy, and style relative to earlier sequential outputs, although gains vary by language and quality dimension.The full human evaluation is split by language direction.
- Human evaluation: At budget 30, parallel sampling often achieves higher accuracy than sequential sampling.The comparison concerns sequential versus parallel sampling with a selector.
- Context ablation: Context restriction substantially improves selector-free sequential sampling, while selector-based runs show the same pattern with smaller differences.The ablation compares full context, context-5, context-2, and context-1 sliding windows.
- Sampling and selectors: Sequential sampling explores output spaces more effectively than parallel sampling, especially under limited budgets, and this finding remains consistent with MetricX24 as selector.The alternate-selector result is reported in Figure 20.
- Sampling and temperature: Temperature 1.0 is near optimal for Best-of-N sampling, whereas selector-free Translate Again performs better at lower temperatures.The latter reflects a higher-precision, lower-diversity regime.
- Diversity: Sequential sampling produces fewer duplicates than parallel sampling in earlier rounds across language directions.The diversity metric measures similarity of each new sample to previous samples using exact matches.