PaperlayerView exact revision on arXiv

Source-linked AI summary

SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scaling

Jiefeng Chen, Jie Ren, Xinyun Chen, Chengrun Yang, Ruoxi Sun, Jinsung Yoon, Sercan Ö Arık

arXiv:2501.19306v5cs.AIcs.CL

TL;DR

Existing test-time scaling methods can plateau, require additional task-specific models, or struggle to improve through sequential refinement. SETS combines parallel sampling with iterative self-verification and self-correction, and experiments report stronger scaling across diverse tasks and models. Its effectiveness remains tied to the underlying self-critique and self-correction abilities of the LLM.

  • Problem

    Parallel test-time scaling can quickly saturate and rely on task-specific reward models, while sequential self-refinement can struggle to improve after a few rounds.

  • Method

    SETS combines parallel and sequential scaling by sampling responses, self-verifying them, and iteratively self-correcting them without training additional models.

  • Results

    SETS consistently outperforms baselines across challenging planning, reasoning, math, and coding tasks and shows increasing returns as test-time computation increases.

  • Takeaways & Limitations

    SETS provides a unified approach for scaling test-time computation using LLM self-verification and self-correction across diverse models and tasks.

  • Takeaways & Limitations

    SETS efficacy is directly tied to the LLM’s self-critique and self-correction capabilities.

Abstract

from arXiv · show

Recent advancements in Large Language Models (LLMs) have created new opportunities to enhance performance on complex reasoning tasks by leveraging test-time computation. However, existing scaling methods have key limitations: parallel methods like repeated sampling are often inefficient and quickly saturate, while sequential methods like SELF-REFINE struggle to improve after a few rounds. Although combining these approaches shows promise, current methods require fine-tuned reward and revision models. This paper proposes Self-Enhanced Test-Time Scaling (SETS), a simple yet effective approach that overcomes these limitations by strategically combining parallel and sequential techniques and fully leveraging LLMs' self-improvement abilities. SETS exploits the inherent self-verification and self-correction capabilities of LLMs, unifying sampling, verification, and correction within a single framework. This facilitates efficient and scalable test-time computation for enhanced performance on complex tasks without any model training. Our comprehensive experimental results on challenging benchmarks spanning planning, reasoning, math, and coding demonstrate that SETS achieves significant performance improvements and more advantageous test-time scaling behavior than the alternatives.

1. Introduction

Existing test-time scaling methods face efficiency and scalability limits: repeated sampling quickly plateaus, while sequential self-refinement and task-specific reward models constrain broader use. SETS addresses this by combining parallel and sequential scaling through LLM self-verification and self-correction.

  • Repeated sampling often plateaus as test-time compute increases and may require task-specific reward models, adding training overhead.
  • Sequential methods such as SELF-REFINE can struggle to improve after a few rounds, motivating alternative scaling strategies.
  • SETS combines parallel and sequential scaling with Sampling, Self-Verify, and Self-Correct operations without training additional models.
  • SETS is evaluated on planning, reasoning, math, and coding benchmarks, where it maintains higher effectiveness and experiences less fall-off in performance gains than alternatives.
  • The reported experiments show up to 10.9% accuracy improvement across planning, reasoning, math, and coding benchmarks with non-thinking and thinking models.

2. Related Work

Related work studies parallel and sequential test-time scaling, self-verification, self-correction, and compute allocation across model sizes. SETS builds on these directions by using LLM self-verification and self-correction within a combined scaling framework.

  • Test-time scaling methods mainly use parallel sampling with aggregation or sequential refinement to improve LLM responses.
  • Traditional verification approaches often train additional verifiers, whereas recent work shows that LLMs can self-verify their outputs.
  • LLM self-correction can use external feedback, self-feedback, or oracle evaluation, but earlier findings report difficulty correcting responses without external feedback.
  • Prior work examines trade-offs between model size and test-time compute using strategies such as greedy search, majority voting, and Best-of-N.
  • The paper studies how test-time scaling behavior varies across models and query difficulty when self-verification and self-correction are used.

3. Method

SETS combines sampling, self-verification, and self-correction: it generates multiple responses, iteratively improves each response using self-feedback, and applies majority voting to select the final solution.

  • The framework uses three operations—Sampling, Self-Verify, and Self-Correct—each associated with a dedicated prompt.
  • Self-Verify produces feedback indicating whether a proposed solution is correct, and a judgement function maps that feedback to a correctness indicator.
  • SETS samples m initial responses and applies Self-Verify and Self-Correct iteratively for up to n rounds to improve each response.
  • After refining all sampled responses, SETS uses majority voting to select the final solution.
  • The framework can use exact matching or an LLM judge to determine response equivalence; the experiments use exact matching because benchmark answers are well structured.
  • SELF-REFINE is a special case of SETS when m=1, while SETS is designed to scale more effectively when larger compute budgets are available.

4. Experiment

The experiments evaluate SETS across diverse models and challenging planning, reasoning, math, and coding benchmarks, using compute-optimal scaling curves and controlled analyses. SETS generally improves accuracy and scaling efficiency over sampling and SELF-REFINE baselines, while results vary with model capabilities and configuration.

  • Setup: Experiments cover six complex-reasoning datasets and both proprietary and open-source LLMs, including non-thinking and thinking models.The datasets span planning, reasoning, mathematics, and coding tasks.
  • Setup: SETS combines Sampling, Self-Verify, and Self-Correct, whereas the baselines omit one or more of these components and require no additional model training or external reward models.The method compares against SELF-REFINE and Best-of-N variants under shared prompting conditions.
  • Improved Test-time Scaling with SETS: SETS consistently outperforms the baselines across benchmarks, with accuracy gains increasing as test-time compute grows, while majority-vote sampling typically saturates quickly.These scaling curves use hyperparameter settings selected to maximize performance within a compute budget.
  • Improved Test-time Scaling with SETS: SETS improves accuracy across all datasets, unlike self-verification or self-evaluation variants whose gains over majority voting are task-dependent; the pattern also holds when cost is measured by API calls.This comparison evaluates alternative ways of aggregating or refining sampled responses.
  • Impact of Different LLMs: SETS generally outperforms baselines across additional LLMs, with the strongest gains associated with models having stronger self-verification and self-correction capabilities.The paper evaluates Gemini, Claude, and Qwen models and separately measures verification and correction performance.
  • The Effect of Self-Correction Rounds: At a fixed compute budget, increasing SETS correction rounds generally improves accuracy, with diminishing returns after n=4 for Trip Planning and Meeting Planning.The authors suggest m=50 and n=4 as practical SETS settings.
  • The Effect of Temperature for SETS: Using temperature 0.7 for Sampling, Self-Verify, and Self-Correct generally performs better than greedy verification and correction, potentially by exploring more reasoning paths.The paper connects this diversity with the final majority-voting mechanism.
  • Non-thinking Mode with SETS vs. Thinking Mode: SETS amplifies a model’s existing reasoning capability rather than creating it: a non-thinking model with SETS did not match a stronger thinking model using BoN+Majority Vote under a fixed token budget.Applying SETS to the thinking mode still produced substantial gains.

5. Conclusions

SETS combines sampling with iterative self-verification and self-correction, outperforming repeated sampling and SELF-REFINE while showing increasing returns as computation grows. Its scope remains bounded by dependence on self-improvement capabilities, efficiency needs, objectively verifiable reasoning tasks, and text-only evaluation.

  • Contributions: SETS integrates parallel sampling with sequential self-verification and self-correction, surpassing repeated sampling and SELF-REFINE across challenging planning, reasoning, math, and coding tasks.The framework samples initial responses, iteratively refines them, and uses majority voting for the final solution.
  • Results: SETS demonstrates increasing returns as test-time computation increases across planning, reasoning, math, and coding tasks.
  • Limitations: SETS efficacy is directly tied to the self-critique and self-correction capabilities of the underlying LLMs.The authors identify improving these capabilities as a key future-work priority.
  • Limitations: The current work focuses on reasoning tasks with objectively verifiable answers, while future work targets summarization, tool use, and multimodal benchmarks.Extending beyond objectively verifiable answers may require aggregation methods more sophisticated than majority voting.

A. Datasets

The experiments use six datasets spanning planning, reasoning, mathematics, and coding, with structured answers that support exact-match or rule-based evaluation. The study pairs these benchmarks with simple, standardized prompts for sampling, self-verification, and self-correction.

  • Datasets: The evaluation covers six datasets: Trip Planning, Meeting Planning, LiveBench Reasoning, MATH 500, AIME 2024-2025, and LiveCodeBench TestOutputPred.
  • Planning: NATURAL PLAN includes Trip Planning and Meeting Planning, which provide five-shot exemplars, controlled difficulty variables, and long-form multi-step ground-truth solutions.
  • Reasoning: LiveBench Reasoning contains spatial, zebra_puzzle, and web_of_lies_v2 tasks, each with 50 test examples, and is designed with contamination control and objective evaluation in mind.
  • Mathematics: MATH 500 is a 500-problem subset of the 12,500-problem MATH benchmark, while AIME 2024-2025 contains problems from the AIME competitions.
  • Coding: LiveCodeBench TestOutputPred contains 442 examples and evaluates test-output prediction within a broader contamination-free coding benchmark.
  • Prompts: The method uses minimal, standardized, mostly zero-shot prompts for Sampling, Self-Verify, and Self-Correct across tasks and models.Self-Verify prompts ask models to identify constraints and check proposed answers; Self-Correct prompts use solution-analysis pairs to derive revised answers.

D.1. Cost Estimation using Number of API Calls

Using average API calls instead of output tokens to measure computational cost produces the same scaling-law findings.

  • Average API calls provide an alternative computational-cost measure for plotting SETS scaling laws.The corresponding curves use API calls on the x-axis and accuracy on the y-axis.

D.2. Impact of Different LLMs

SETS generally remains effective across proprietary and open-weight models, although performance depends on the model’s ability to generate, critique, and format valid solutions.

  • The experiments evaluate SETS across Claude-3.5-Sonnet, Gemini-2.5-Flash variants, Qwen3-235B-A22B, and Qwen2.5-1.5B-Instruct.The Qwen models were evaluated on a restricted set of benchmarks because of slow inference speeds.
  • SETS consistently and significantly outperforms all baselines across four diverse benchmarks with Qwen3-235B-A22B.The benchmarks span planning, reasoning, math, and coding, demonstrating effectiveness for an open-weights model.
  • SETS benefits smaller Qwen2.5-1.5B-Instruct on LiveBench Reasoning and LiveCodeBench TestOutputPred but underperforms BoN+Majority Vote on MATH 500.All methods fail on complex Trip Planning because the base model lacks the capacity to generate valid solutions.
  • Poor instruction-following can prevent successful answer parsing and limit SETS’s advantage over baselines.This issue occurred for Gemini-2.5-Flash-Lite-Thinking on Trip Planning, where incorrectly formatted responses became “None” values.
  • Increasing self-verification samples improves verification performance, but the gain typically saturates quickly.Verification performance is measured by AUROC between verification scores and solution correctness.

D.6. Performance under the Oracle Setting

The oracle setting compares SETS and Best-of-N after sampling, using ground-truth references to select the final solution. Under this infeasible evaluation, SETS has a marked advantage on several tasks, especially those with larger and more complex solution spaces.

  • Oracle selection uses ground-truth references to choose the final solution, so it is not feasible in practice.The comparison includes BoN+Oracle and SETS+Oracle.
  • SETS+Oracle selects among up to m·(n+1) generated solutions, while BoN+Oracle selects among m sampled solutions.If all candidates are incorrect, both methods select the first sampled solution.
  • SETS with oracle selection has a marked advantage over BoN+Oracle on Trip Planning, Meeting Planning, and LiveCodeBench TestOutputPred.The advantage is less pronounced on the other evaluated tasks.
  • Figure 15 plots accuracy against average output tokens, with each point representing a hyperparameter setting and its optimal performance at the corresponding cost budget.The curves use subsampling to reduce marker crowding.

D.7. The Impact of Task Difficulty

The study uses controlled task variables to divide NATURAL PLAN instances into easy and hard groups. Figure 16 presents estimated scaling-law curves for SETS and several baselines across these difficulty levels.

  • Task difficulty: Trip Planning tasks with no more than 6 cities are classified as easy, while tasks with more than 6 cities are hard.
  • Task difficulty: Meeting Planning tasks with no greater than 5 people are classified as easy, while tasks with more than 5 people are hard.
  • Scaling curves: The scaling-law plots use average number of output tokens on the x-axis and distinguish easy and hard task curves.
  • Scaling curves: Figure 16 compares SETS with BoN+Majority Vote, BoN+Self-Verify, BoN+Self-Eval, and SELF-REFINE.

D.8. Performance under fixed hyperparameters

Under fixed hyperparameters, SETS generally outperforms Best-of-N baselines at comparable computational budgets, while cost-based scaling results match token-based findings. Additional analyses show task-dependent confidence voting, capability amplification rather than capability creation, and self-verification failures from hallucinated reasoning.

  • SETS generally demonstrates significantly superior performance over Best-of-N baselines under comparable computational budgets.Experiments use fixed hyperparameters and report mean and standard deviation across three repetitions.
  • Average-price scaling curves reproduce the findings obtained when average output tokens measure computational cost.Figure 17 plots accuracy against average price for hyperparameter settings.
  • SETS amplifies a model’s existing reasoning capability rather than bridging the fundamental gap between non-thinking and thinking modes.Under a fixed token budget, SETS in non-thinking mode does not match thinking mode with Best-of-N majority voting, while applying SETS to thinking mode yields substantial gains.
  • Confidence-weighted voting improves over majority voting on most benchmarks, but majority voting remains stronger on Meeting Planning and AIME 2024-2025.The results indicate that self-verification confidence is a task-dependent heuristic.
  • Qualitative analyses attribute self-verification failures primarily to hallucinations that produce factually incorrect or nonsensical reasoning.Examples include both false-negative and false-positive verification outcomes in trip-planning tasks.
  • A MATH 500 example shows self-verification correcting an incorrect proposed answer by checking divisibility and minimality conditions.The demonstrated target is 2220, while 202020 is rejected because it is larger despite satisfying the digit constraints.
Loading 2501.19306v5…