PaperlayerView exact revision on arXiv

Source-linked AI summary

The Art of Scaling Test-Time Compute for Large Language Models

Aradhye Agarwal, Ayan Sengupta, Tanmoy Chakraborty

arXiv:2512.02008v1cs.CL

TL;DR

The paper addresses the lack of systematic evidence about how TTS strategies interact with model type and problem difficulty. It evaluates competing strategies across models and benchmarks, finding that no strategy is universally best and that effective choices depend on reasoning horizon, difficulty, and compute budget. The resulting framework provides a practical recipe for model-aware inference-time scaling.

  • Problem

    The best TTS strategy remains unclear because prior evidence does not systematically account for model variation, problem difficulty, and compute availability.

    Source: §1 · p. 2

  • Method

    The study evaluates TTS strategies across reasoning and non-reasoning models, reasoning benchmarks, and API-based inference settings.

    Source: §2.2 · pp. 3–4, §2.3 · p. 4

  • Results

    No single TTS strategy is universally optimal; short-horizon models favor concise traces, while long-horizon models benefit from longer reasoning on harder problems.

    Source: §1 · p. 2, §3.2 · p. 7, §6 · p. 9

  • Takeaways & Limitations

    TTS should be selected using a practical recipe that jointly considers model type, problem difficulty, and available compute budget.

    Source: §5 · p. 8, §6 · p. 9

    The analysis is restricted to API-friendly TTS strategies because inference uses API-based model access.

    Source: §2.2 · p. 3

Abstract

from arXiv · show

Test-time scaling (TTS) -- the dynamic allocation of compute during inference -- is a promising direction for improving reasoning in large language models (LLMs). However, a systematic comparison of well-known TTS strategies under identical conditions is missing, and the influence of model type and problem difficulty on performance remains unclear. To address these gaps, we conduct the first large-scale study of TTS, spanning over thirty billion tokens generated using eight open-source LLMs (7B to 235B parameters), across four reasoning datasets. We observe three consistent trends: (1) no single TTS strategy universally dominates; (2) reasoning models exhibit distinct trace-quality patterns across problem difficulty and trace length, forming short-horizon and long-horizon categories; and (3) for a given model type, the optimal TTS performance scales monotonically with compute budget. Based on these insights, we provide a practical recipe for selecting the best TTS strategy, considering problem difficulty, model type, and compute budget, providing a practical guide to effective inference-time scaling.

1 Introduction

Test-time scaling improves LLM reasoning by allocating more inference compute, but the best strategy remains unclear across models and tasks. This study argues that model training, reasoning horizon, problem difficulty, and compute budget should guide strategy selection.

Prior findings

  • Longer reasoning can amplify errors and degrade accuracy, so more inference compute is not universally beneficial.Prior studies report that extended traces may reinforce incorrect behaviors, misalign reasoning paths, and create safety concerns.Source: §1 · p. 2

Motivation

  • The study revisits TTS using newer model families because prior work did not account for model variation and relied on older reasoning models.The authors specifically include GPT-OSS and Qwen3, finding that compute-performance relationships vary across model families.Source: §1 · p. 2

Reasoning horizons

  • Reasoning models form short-horizon and long-horizon categories with different trace-quality patterns across problem difficulty.Short-horizon models favor concise reasoning, whereas long-horizon models can benefit from sustained reasoning on harder problems.Source: §1 · p. 2
  • Post-training methods appear linked to reasoning horizon: GRPO-like training commonly yields short-horizon models, while alternatives such as GSPO support longer traces.Qwen3 is presented as a long-horizon model post-trained with GSPO rather than GRPO.Source: §1 · p. 2

Practical motivation

  • Effective TTS strategy selection must account jointly for model type, problem difficulty, and available compute.The paper frames this model-aware perspective as the basis for principled inference-time scaling.Source: §1 · p. 2

2 Preliminaries

The paper organizes TTS methods by how they allocate inference computation and evaluates them across model types, reasoning benchmarks, metrics, and difficulty measures. These preliminaries motivate a model- and difficulty-aware comparison of scaling strategies.

2.1 Test-time scaling methods

  • TTS methods include parallel, sequential, hybrid/meta, and internal-compute paradigms, with no class universally optimal.Parallel methods aggregate sampled traces, sequential methods extend or revise reasoning, hybrid methods adapt between strategies, and internal methods adjust computation within inference.Source: §2.1 · pp. 2–3

2.2 Models

  • The analysis compares reasoning and non-reasoning models, including DeepSeek-R1, GPT-OSS-120B, Qwen3-32B, DAPO-32B, Qwen3-235B-Instruct, and DeepSeek-Chat.The evaluation includes both reasoning-centric and general-purpose models, while API-based inference restricts the study to API-friendly strategies.Source: §2.2 · pp. 3–4

2.3 Datasets

  • AIME measures symbolic and arithmetic reasoning, while GPQA Diamond measures graduate-level conceptual and factual reasoning across science domains.Together, the benchmarks cover mathematical, symbolic, conceptual, and knowledge-grounded reasoning.Source: §2.3 · p. 4

2.4 Metrics

  • Accuracy is the proportion of traces whose final prediction matches the ground truth, while token metrics distinguish total compute from sequential latency.Total tokens count all generated traces; sequential tokens estimate the minimum possible latency under parallelizable generation.Source: §2.4 · pp. 4–5

2.5 Measuring Problem Difficulty

  • Across datasets, both reasoning and non-reasoning models generate more tokens on harder problems, although longer traces alone do not guarantee higher quality.Problem difficulty can be estimated through accuracy or average generated tokens because the two measures are correlated.Source: §2.5 · p. 5, Figure 3 · p. 5

3 Results

Beam search provides no reliable compute scaling, while trace-quality relationships differ by model horizon and problem difficulty.

3.1 Beam search shows inverse or no scaling

  • Beam search accuracy degrades or remains unchanged as beam size increases, despite higher token consumption and compute.This inverse or flat scaling appears across reasoning-focused datasets and model families.Source: §3.1 · p. 6

3.2 Correlation of trace length with quality

  • Table 1 classifies reasoning models as short-horizon or long-horizon by comparing accuracy for short and long traces across task difficulties.Tasks are split into easy and hard using median difficulty, while trace lengths use model-specific medians; analyses restrict attention to datasets containing both trace lengths.Source: §3.2 · p. 6, Table 1 · p. 6
  • For every trace-length bucket, easy problems achieve higher accuracy than hard problems across all models.Problem difficulty is defined through aggregated accuracy, so harder questions naturally have lower correctness rates.Source: §3.2 · p. 6
  • Figures 4 and 5 compare accuracy with token usage for FFS-k and LFS@N variants across model families.Figure 4 colors FFS variants by k and sizes markers by N; Figure 5 colors LFS variants by N and sizes markers by k.Source: Figure 4 · p. 7, Figure 5 · p. 7
  • Short-horizon models R1, QwQ-32B, and DAPO-32B favor shorter traces for both easy and hard problems.DAPO-32B exhibits a length-bias level close to R1, suggesting limited mitigation relative to GRPO under this evaluation.Source: §3.2 · p. 7
  • Qwen3-32B and GPT-OSS-120B prefer shorter traces on easy problems but longer traces on hard problems.This contrasts with the consistent short-trace preference observed in short-horizon models.Source: §3.2 · p. 7

4 Analysis

The analysis finds that optimal TTS choices scale with available compute, while LFS is dominated by majority voting and FFS behavior depends on model horizon.

4 Analysis

  • The optimal TTS strategy scales with increasing compute budget across the evaluated model types.Figures 4 and 5 show this pattern for FFS-k@N and LFS-k@N.Source: §4 · pp. 7–8
  • LFS-k@N is always suboptimal to MV@N because maximum performance at fixed compute occurs when k=N.When k=N, LFS becomes majority voting over all N traces, while using the same token budget.Source: §4 · p. 8
  • FFS performance improves with larger k across model types, but the effect of N at fixed k differs by model family.Short-horizon models favor larger N, whereas long-horizon and non-reasoning models face performance–compute tradeoffs with opposite control directions.Source: §4 · p. 8

5 The Recipe

The recipe selects TTS strategies from model horizon, task difficulty, and compute budget, although the final strategy is independent of task difficulty.

5 The Recipe

  • The decision matrix makes TTS selection depend on model family, problem difficulty, and available compute budget.It provides actionable guidance for choosing among FFS, LFS, MV@N, and simple decoding.Source: §5 · p. 8
  • For short-horizon models, use FFS with k=1 under low compute and k=N under high compute, while choosing the largest feasible N.These models prefer shorter traces across difficulty levels, and larger N improves performance for fixed k.Source: §5 · p. 8
  • For long-horizon models on hard tasks, use k=N and increase N with compute; on easy tasks, use FFS with N=k.Low compute reduces these choices to simple decoding, while high compute uses majority voting with larger samples.Source: §5 · pp. 8–9
  • The final recipe is independent of task difficulty despite distinct trace preferences across easy and hard settings.Difficulty changes the observed behavior, but not the strategy selected in the final decision matrix.Source: §5 · p. 9

6 Conclusion

The study finds no universally optimal TTS strategy and instead recommends model-aware inference scaling. Effective choices depend on training methodology, problem difficulty, and compute budget.

6 Conclusion

  • No single TTS strategy is optimal for all LLMs and tasks.The most effective approach depends on the interaction among training methodology, problem difficulty, and available compute.Source: §6 · p. 9
  • Short-horizon models favor concise traces, whereas long-horizon models benefit from longer reasoning on harder problems and concise reasoning on easier problems.The conclusion frames these differences as model-family-specific behavior requiring model-aware inference.Source: §6 · p. 9
  • Beam search is consistently suboptimal for complex reasoning.The paper therefore supports nuanced inference-time scaling rather than a universal strategy.Source: §6 · p. 9

A Hyperparameters

The experiments use a fixed set of decoding hyperparameters summarized across models and datasets in Table 3.

B Results

The results show complementary trade-offs among beam search, majority voting, first-finish search, and last-finish search in accuracy and compute cost.

  • MV is the most consistent accuracy-oriented method but incurs the largest token costs, often an order of magnitude higher than FFS.Source: B Results · pp. 12–13
  • BS occasionally achieves top accuracy on individual rows but is typically expensive.Source: B Results · p. 12
  • FFS commonly reduces MV token usage by tens of percent up to approximately 90%, but its accuracy varies substantially by model.Source: B Results · p. 13
  • LFS has mixed reliability, matching or surpassing MV on some symbolic tasks while underperforming on several models.Source: B Results · p. 13

Model-family comparison

Method preferences vary across model families and tasks, so no single decoding strategy is uniformly best.

  • Qwen-derived models often favor MV and LFS on math tasks, while FFS provides the largest efficiency gains without uniformly winning accuracy.Source: Model-family comparison · p. 13
  • For the DeepSeek family, MV is the most stable high-accuracy choice, while FFS sharply reduces compute with variable accuracy trade-offs.Source: Model-family comparison · p. 13
  • For GPT-OSS-120B and QwQ-32B, MV or LFS usually leads accuracy, whereas FFS has mixed effects across models.Source: Model-family comparison · p. 13
  • GPQA shows small accuracy spreads across methods, while AIME variants are more sensitive and usually favor MV or LFS for symbolic reasoning.Source: Model-family comparison · p. 13
  • MV is safest at higher compute cost, FFS is attractive under compute constraints, and LFS can be useful for symbolic problems.Source: Model-family comparison · p. 13

C Model-wise Plots

Figures 6 and 7 plot accuracy against token usage for FFS and LFS across the eight evaluated models.

Loading 2512.02008v1…