Source-linked AI summary
A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, Irwin King, Xue Liu, Chen Ma
TL;DR
Test-time scaling has become important as pretraining scaling slows and lacks a unified account of its rapidly expanding methods. This paper develops a four-dimensional survey taxonomy, reviews methods and applications, and provides deployment guidance. It identifies developmental trends, open challenges, and future directions while noting interpretability and deployment constraints.
Problem
Pretraining scaling is resource-intensive and constrained by bounded human data, while TTS research lacks a unified framework for systematic synthesis, comparison, and trend identification.
Method
The paper surveys TTS using a hierarchical taxonomy spanning what to scale, how to scale, where to scale, and how well to scale.
Results
The survey organizes representative methods, application scenarios, and assessment aspects, and distills major developmental trajectories and practical deployment guidelines.
Takeaways & Limitations
The framework supports structured classification, comparison, extensibility, and more targeted analysis of TTS challenges and future research directions.
Takeaways & Limitations
Internal scaling is difficult to diagnose or regulate because it lacks clear intermediate checkpoints, and it requires substantial tuning resources that can limit practitioner access.
Abstract
from arXiv · showhide
As enthusiasm for scaling computation (data and parameters) in the pretraining era gradually diminished, test-time scaling (TTS), also referred to as ``test-time computing'' has emerged as a prominent research focus. Recent studies demonstrate that TTS can further elicit the problem-solving capabilities of large language models (LLMs), enabling significant breakthroughs not only in specialized reasoning tasks, such as mathematics and coding, but also in general tasks like open-ended Q&A. However, despite the explosion of recent efforts in this area, there remains an urgent need for a comprehensive survey offering a systemic understanding. To fill this gap, we propose a unified, multidimensional framework structured along four core dimensions of TTS research: what to scale, how to scale, where to scale, and how well to scale. Building upon this taxonomy, we conduct an extensive review of methods, application scenarios, and assessment aspects, and present an organized decomposition that highlights the unique functional roles of individual techniques within the broader TTS landscape. From this analysis, we distill the major developmental trajectories of TTS to date and offer hands-on guidelines for practical deployment. Furthermore, we identify several open challenges and offer insights into promising future directions, including further scaling, clarifying the functional essence of techniques, generalizing to more tasks, and more attributions. Our repository is available on https://github.com/testtimescaling/testtimescaling.github.io/
1 Introduction
The paper surveys test-time scaling as a response to slowing, resource-intensive pretraining scaling and the bounded availability of human data. It introduces a unified four-axis framework, organizes methods and applications, and derives practical guidance, challenges, and future directions.
- Motivation: Pretraining progress has slowed because scaling is resource-intensive and human data is bounded, motivating efforts to elicit more intelligence from LLMs at test time.
- Research Gap: The survey addresses the lack of a unified framework for synthesizing insights, comparing techniques, and identifying consistent trends in test-time scaling.
- Framework: Its taxonomy organizes TTS across four axes: what to scale, how to scale, where to scale, and how well to scale.
- Analysis and Guidance: The paper surveys representative methods, maps applications and performance dimensions, and provides guidelines for research and deployment.
- Future Directions: It identifies challenges and future directions including advancing scaling, clarifying technique essence, and extending TTS across tasks and domains.
2 What to Scale
The paper classifies what is expanded during test-time inference and distinguishes parallel, sequential, hybrid, and internal scaling strategies. These approaches trade off breadth of solution coverage, iterative reasoning, and autonomous computation allocation.
- Overview: “What to scale” denotes the specific form of test-time computation expanded or adjusted to improve LLM performance during inference.
- Parallel Scaling: Parallel scaling generates multiple candidate responses and aggregates them, with effectiveness depending on solution coverage and aggregation quality.
- Parallel Scaling: Parallel scaling includes repeated sampling from one model and sampling across multiple models, with temperature changes and prompt rephrasing used to increase diversity.
- Sequential Scaling: Sequential scaling iteratively updates intermediate solution states, supporting step-by-step reasoning, refinement, decomposition, and self-correction.
- Hybrid Scaling: Hybrid scaling combines parallel exploration with sequential filtering or refinement, as illustrated by tree-, graph-, forest-, and multi-agent reasoning methods.
- Internal Scaling: Internal scaling trains models to autonomously allocate computation and control halting through internal states, without external prompts or multi-call orchestration.
3 How to Scale
The survey organizes scaling methods into training-based and inference-based approaches. Inference-based TTS combines stimulation, verification, search, and aggregation to allocate computation during deployment.
- Training-based Approaches: Training-based approaches activate test-time reasoning by tuning model parameters through SFT, RL, or preference optimization.SFT can teach structured reasoning patterns, while RL and preference methods target reasoning performance and exploration.
- Training-based Approaches: Distillation transfers reasoning capabilities from stronger models or ensembles into target models through supervised learning.A reported 32B model trained on curated outputs from a top-tier reasoner solved competition-level mathematics nearly as well as its teacher.
- Inference-based Approaches: Inference-based approaches dynamically adjust deployment computation through stimulation, verification, search, and aggregation.These components encourage more candidate generation, assess candidate quality, explore solution spaces, and consolidate outputs.
- Stimulation: Stimulation increases computation by eliciting longer or multiple samples through prompt, decoding, latent, self-repetition, and mixture-of-model strategies.The strategies modify instructions, decoding behavior, hidden-state reasoning, repeated prompting, or the number and diversity of participating models.
- Verification and Aggregation: Verification evaluates candidate correctness or consistency, while aggregation produces the final answer through selection or fusion.Verification may use external tools, rethinking, or multiple criteria; aggregation selects a candidate or combines several candidates.
4 Where to Scale
The survey maps TTS applications across reasoning-intensive, agentic, general, open-ended, knowledge-intensive, evaluation, and multimodal tasks. These settings differ in their reasoning demands, feedback or verification requirements, and benchmark coverage.
- Reasoning-Intensive Tasks: Reasoning-intensive applications include mathematics, programming, strategic games, science, and medicine, each requiring structured reasoning and domain-specific verification.Representative benchmarks span MiniF2F and MATH-500, Codeforces and SWE-bench, SysBench, GPQA Diamond, and MedQA.
- Agentic Tasks: Agentic TTS scales deliberate design choices, emergent social behavior, or agent-environment interaction.Agentic benchmarks such as WebShop, WebArena, SciWorld, and TextCraft evaluate planning, tool use, adaptive decisions, and iterative refinement.
- General and Open-Ended Tasks: General and open-ended tasks evaluate broad reasoning, creativity, diversity, quality, coherence, and subjective output assessment.Examples include AGIEval, MMLU-Pro, AlpacaEval2.0, ArenaHard, and IF-Eval.
- Knowledge-Intensive Tasks: Knowledge-intensive tasks emphasize retrieval-augmented reasoning, iterative verification, and multi-source factual synthesis to reduce hallucinations.SimpleQA, C-SimpleQA, and FRAMES assess factual correctness and retrieval-based reasoning.
- Evaluation Tasks: Evaluation tasks use LLMs as judges or Generative Reward Models to assess candidate responses comprehensively and reliably.TTS evaluation methods include comparing against crowd-generated responses, generating tailored evaluation plans, and learning evaluation principles with RL.
- Multimodal Tasks: Multimodal tasks require cross-modal integration, iterative reasoning across visual and textual inputs, and robust verification under ambiguity.Benchmarks include MMMU, MathVista, MathVision, CMMaTH, and PGPS9K.
5 How Well to Scale
The survey evaluates test-time scaling across Performance, Controllability, Scalability, and Efficiency, covering correctness, steerability, resource use, and reasoning quality. It also formalizes efficiency as the trade-off between solution quality and computational cost, while highlighting redundancy, underthinking, and overthinking.
- Test-time scaling evaluation spans four dimensions: Performance, Controllability, Scalability, and Efficiency.These dimensions capture correctness, output control, resource behavior, and reasoning effectiveness.
- Performance: Pass@1 measures first-attempt correctness, while Pass@k measures whether at least one of k sampled outputs is correct.Pass@1 is common in mathematical reasoning and coding; Pass@k is widely used in program synthesis and theorem proving.
- Performance: Cons@k evaluates majority-vote correctness across independently sampled outputs, complementing Pass@1 by measuring the benefit of multiple samples.Larger k values typically improve answer stability and accuracy, according to the survey.
- Efficiency: Reasoning efficiency is defined as expected solution quality divided by computational cost, exposing when longer reasoning fails to yield proportional gains.Quality may include accuracy, exact match, or creativity; cost may include tokens, FLOPs, or latency.
- Efficiency: Reasoning inefficiency includes redundancy, underthinking, and overthinking, all of which can increase computation without proportional gains in solution quality.These patterns respectively involve repeated reasoning, prematurely abandoned directions, and excessive revisiting or verification.
- Efficiency: ETS reduces KV-cache size by up to 1.8× versus REBASE and delivers 1.4× faster inference on NVIDIA H100 GPUs without compromising accuracy.The reduction comes from sharing KV caches across trajectories.
- Controllability: k–ϵ controllability measures whether a model can produce a target output with probability at least 1 −ϵ using a prompt of length at most k.Across evaluated k and ϵ values, over 97% of WikiText targets were reachable with prompts of at most 10 tokens and ϵ ≤0.05.
6 Organization and Trends in Test-time scaling
The survey organizes test-time scaling literature into a taxonomy and traces a progression from structured inference to automated supervision and reinforcement learning. It concludes that methods are complementary, no single scaling solution fits every problem, and inference- and tuning-based approaches increasingly overlap.
- The survey decomposes inference scaling literature using a taxonomy and presents common combinations of techniques in Table 5.The framework is intended to organize technically different approaches within a shared structure.
- From 2022 to 2023, researchers emphasized structured inference for guiding LLMs toward more complex solutions.The survey describes this as the early stage of the field’s evolutionary path.
- In 2024, PRM and MCTS enabled automatic supervision of intricate reasoning trajectories, producing richly annotated fine-tuning data and improving TTS performance.The passage presents these methods as a subsequent development in the field’s progression.
- TTS techniques are complementary: R1 requires an SFT-based warmup through rejection sampling, and stronger scaling therefore requires systematic integration.The survey also recommends combining synthesized CoT approaches and structured inference within RL frameworks.
- No single scaling solution works for all problems, motivating increasing attention to optimal-scaling solutions.The survey identifies this as a major trend in current research.
- The boundary between inference-based and tuning-based approaches is blurring as inference capabilities are distilled into models and exploited across training and inference.The target of scaling can change across different stages of the pipeline.
7 A Hand-on Guideline for Test-time Scaling
The survey translates its taxonomy into practical guidance for deploying TTS, emphasizing task fit, technical pathways, evaluation criteria, and efficiency trade-offs.
- Task Selection: TTS benefits traditional reasoning tasks and shows promising gains in open-ended, medical, and legal applications, though open-ended outputs are harder to evaluate quantitatively.
- Practical Deployment Paths: Three essential TTS pathways are deliberate inference-time reasoning, imitation of complex reasoning trajectories, and RL-based incentivization.
- Practical Deployment Paths: These pathways are complementary and can be integrated into frontier-level strategies rather than treated as mutually exclusive alternatives.The survey cites R1 as combining rejection-sampling SFT warmup with RL and synthesized CoT or structured inference.
- Evaluation: Evaluation should prioritize task-aware accuracy while also considering efficiency, robustness, safety, bias, and interpretability.Efficiency captures the practical trade-off between performance and computational cost.
- Efficiency Trade-offs: Internal scaling can be more efficient because it prompts the language model once, but tuning it requires substantial resources that limit practitioner accessibility.
8 Challenges and Opportunities
The survey identifies open challenges in scaling, technique interpretation, evaluation, and generalization, and proposes directions for making TTS more adaptive, reliable, efficient, and broadly applicable.
- More Scaling is the Frontier: Parallel scaling improves reliability through multiple responses, but diminishing returns emerge as coverage saturates.
- More Scaling is the Frontier: Future parallel methods should expand coverage intelligently through diverse reasoning paths and real-time verification.
- More Scaling is the Frontier: Sequential scaling must preserve coherence and prevent error accumulation through structured refinement and verification between iterations.
- More Scaling is the Frontier: Verification before proceeding can turn sequential scaling into a self-correcting paradigm for goal-directed, long-horizon thinking.
- More Scaling is the Frontier: Hybrid scaling should dynamically select strategies for different queries and extend to multi-agent debate, argumentation, or negotiation.
- Clarifying the Essence of Techniques: Internal scaling must allocate extra reasoning selectively, because overthinking simple tasks or underthinking complex ones eliminates dynamic-computation benefits.
- Clarifying the Essence of Techniques: Internal scaling also faces risks of logical drift and hallucination, while lacking external guidance and clear intermediate checkpoints for diagnosis.
- Clarifying the Essence of Techniques: Future research should clarify how SFT, RL, reward modeling, and chain-of-thought properties contribute to TTS and how they should be combined.
9 Conclusion
The conclusion presents TTS through a hierarchical taxonomy, pairs it with practical guidance, and uses the framework to organize future research opportunities.
- The survey decomposes TTS hierarchically to support conceptual understanding and identification of individual contributions.
- Its hands-on guideline is aligned with the taxonomy dimensions, while the framework organizes trends, challenges, and opportunities for future research.
A Detailed Outcome Verification Methods
Outcome verification techniques operate during inference by generating multiple solutions and using a proposer–verifier framework, unlike training-time methods.
- Outcome verification methods act on the fly during inference rather than during training.
- These methods often generate multiple solutions and apply a proposer–verifier framework to assess outcomes.
A.1 Verifier Model-Based Scoring
Verifier models score candidate outputs according to expected correctness or quality, enabling selection among alternatives. Variants include pairwise comparison, weighted voting, and LLM-based verification.
- Verifier models score each candidate according to its expected correctness or quality.They are typically trained using human feedback or supervised data.
- Pairwise comparison verifiers select a winner by comparing candidates against one another.
- Weighted voting systems combine candidate outputs using verifier scores.
- LLM-based verifiers are another variant of verifier-model scoring.
A.2 Self-Consistency and Voting Mechanisms
Self-consistency generates multiple reasoning chains and selects answers through voting or confidence-aware aggregation. Verification can also use execution, retrieval, or rule-based checks to reject or revise outputs.
- Self-consistency generates multiple independent reasoning chains and selects the most frequent answer by majority voting.The approach assumes that convergence among several chains makes the shared answer more likely to be correct.
- Confidence scores and soft-voting schemes can reduce noise in individual reasoning outputs.
- Execution-based verification checks candidate programs on test cases and validates mathematical answers against equations or constraints.These checks provide an external assessment of the language model’s internal reasoning.
- Retrieval-based fact-checking searches for evidence supporting an answer’s claims and can reject or revise answers contradicted by retrieved documents.
- Rule-based filters automatically reject outputs that violate predefined requirements for form or content.Dialogue systems can regenerate or adjust outputs that match forbidden-answer lists.
B Representative Methods
Representative TTS methods allocate additional inference computation through candidate selection, voting, process evaluation, search, iterative refinement, and branching over reasoning states. These methods vary in how they explore alternatives, assess partial solutions, and use flexible computation budgets.
- Best-of-N: Best-of-N generates N candidate outputs and selects the one with the highest quality score.Its success probability is 1 − (1 − p)^N when each attempt independently succeeds with probability p.
- Majority Voting: Majority voting aggregates independent model or inference predictions and chooses the outcome receiving the most votes.It improves inference reliability without retraining models.
- Process Reward Models: Process Reward Models score complete reasoning trajectories step by step and support process-aware TTS strategies.They are used in methods including stepwise beam search and self-consistency verification.
- Monte Carlo Tree Search: Monte Carlo Tree Search builds a search tree through selection, expansion, simulation, and backpropagation.Its anytime nature allows longer runtime or more rollouts to support deeper search and better decisions.
- Self-Refine: Self-Refine repeatedly generates an answer, produces self-feedback, and refines the output without additional training or fine-tuning.The loop can stop after a predefined iteration count or when output quality converges.
- Tree-of-Thoughts: Tree-of-Thoughts branches at decision points, evaluates partial reasoning states, and explores multiple paths before pruning.Thought generation extends the current state, while a heuristic function estimates the quality of each partial state.