PaperlayerView exact revision on arXiv

Source-linked AI summary

Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck

Davide Romano, Kanak Raj, Jerrod Parker, Daniele Giofrè

arXiv:2608.18931v1cs.CLcs.AI

TL;DR

Test-time scaling has been developed mainly for tasks with reliable verification, leaving its effectiveness on open-ended generation unclear. Evaluating five TTS families across five such benchmarks, the paper finds that exploration improves candidate pools while exploitation is the bottleneck: Fusion recovers only about 40% of available quality.

  • Problem

    TTS lacks systematic evidence on open-ended generation, where nuanced quality rubrics and limited reliable supervision challenge verifier-based exploitation.

  • Method

    The paper compute-normalises and compares five TTS families across five open-ended benchmarks using an exploration–exploitation decomposition of token budgets.

  • Results

    ∼40% is the maximum available headroom recovered by Fusion, while verifier-based methods stagnate or regress and reward-model correlation falls to ∼0.12.

  • Takeaways & Limitations

    The candidate pool is not the bottleneck; converting its high-quality answers into strong final outputs remains the open problem.

  • Takeaways & Limitations

    WritingBench’s apparent Sequential Refinement advantage is confounded by verbosity bias in the benchmark’s evaluation design.

Abstract

from arXiv · show

Test-time scaling (TTS) improves language model outputs by spending additional inference compute - generating multiple candidates, searching over partial sequences, or iteratively refining drafts. These techniques yield large gains on mathematics and code, but have been developed and stress-tested almost exclusively on tasks where verification is straightforward. We conduct the first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing - grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation. The answer depends on which side of that decomposition you examine. Scaling exploration works: the best candidate in the pool improves steadily with compute across all settings. What breaks is exploitation - the step that converts a rich candidate pool into a final output. With state-of-the-art generators, reward models correlate at only $ρ_v \approx 0.12$ with true quality, rendering selection near-random regardless of budget. Tree search amplifies this failure through diversity collapse. Refinement helps on one of five benchmarks; its apparent gains elsewhere are confounded. Only synthesis across candidates (Fusion) consistently improves over single-sample baselines, yet still recovers only ~40% of available quality. The candidate pool is not the bottleneck - choosing from it is.

1 Introduction

This introduction presents the first matched-compute evaluation of five test-time scaling families on open-ended generation and identifies exploitation, rather than exploration, as the central bottleneck. Oracle quality rises with compute, but realised quality stagnates or regresses, with Fusion capturing only ∼40% of available headroom.

  • Evaluation: The study benchmarks Best-of-N, Beam Search, Particle Filtering, Refinement, and Fusion across five open-ended generation benchmarks at matched compute.It provides the first systematic evaluation of TTS in this setting.
  • Central finding: Oracle quality rises with compute, but realised quality stagnates or regresses for verifier-based methods.The oracle measures the true quality of the best candidate in the pool after correcting for judge-noise inflation.
  • Central finding: ∼40% is the maximum available headroom captured by Fusion, measured against the gap between a single-sample baseline and the oracle.None of the tested scaling axes—compute, reward model size, or generator size—closes this gap.
  • Verifier failure: ∼0.12 is the empirical Spearman correlation between reward-model scores and true quality on open-ended generation.Both tested Outcome Reward Models fail similarly, indicating a structural rather than model-specific selection problem.
  • Method diagnosis: 40–60% of independent-sample diversity is lost by Beam Search and Particle Filtering under reward-model guidance, making them worse than parallel Best-of-N.Across the tested families, no exploitation strategy scales reliably; Fusion is the sole method improving over the single-sample baseline on every Qwen3.5 benchmark, while results are mixed on OLMo3.

2 Related Work

Test-time scaling methods comprise five main families, but parallel and search approaches depend on external verifiers to score, select, or guide candidates. Prior evaluation has focused mainly on deterministic, verifiable tasks, leaving open-ended, compute-normalised comparisons and a shared exploration–exploitation framework underdeveloped.

  • Test-time scaling methods and verifiers: TTS methods fall into parallel, search, refinement, fusion, and extended-thinking families, alongside hybrid methods combining or improving these strategies.Parallel methods generate independent candidates; search structures exploration over partial sequences; fusion synthesises across candidates; extended thinking uses one reasoning trajectory.
  • Test-time scaling methods and verifiers: Parallel and search methods depend on external verifiers, with ORMs scoring complete query–completion pairs and PRMs scoring intermediate Chain of Thought steps.Search methods use Process Reward Models to guide exploration over partial sequences.
  • Test-time scaling methods and verifiers: Prior work lacked a systematic comparison of TTS families and a shared framework for measuring exploration against exploitation under a common token budget.The missing framework also concerns how the exploration–exploitation tradeoff degrades when verifier quality drops.
  • Evaluation beyond deterministic tasks: TTS evaluation has focused mostly on math-oriented, deterministic, and verifiable tasks, despite recent movement toward multi-domain benchmarks.The paper addresses this gap through a systematic, compute-normalised comparison on open-ended generation.

3 Theoretical Framework

The framework treats test-time scaling as allocating a fixed token budget between exploration and exploitation. It separates pool quality from the ability to convert that pool into a final response, using headroom capture and verifier correlation to quantify exploitation.

  • Budget decomposition: Every TTS method partitions a fixed budget T into exploration tokens T_e and exploitation tokens T_x, with T = T_e + T_x.This provides the common accounting framework for comparing methods.
  • Quality decomposition: The oracle quality Q*(T) measures the best candidate available in the pool, while realised quality Q(T) measures the gold score of the algorithm’s final output.The distinction isolates candidate generation from final selection or synthesis.
  • Exploitation metric: Headroom capture h is the fraction of exploration headroom realised: h = 1 returns oracle quality, h = 0 matches a single sample, and h < 0 means extra compute hurts.Because oracle quality is the maximum over the full candidate pool, the net exploitation effect is never positive.
  • Verifier quality: For best-of-N selection, headroom capture equals verifier correlation: when ρ_v = 0, more compute provides no benefit, while ρ_v < 0 makes additional compute harmful.This relationship remains usable with judge noise because attenuation cancels, making measured ρ_v a direct predictor of headroom capture.

4 Experimental Setup

The study evaluates five test-time-scaling families on five open-ended benchmarks where exact-match or binary verification is unavailable. It compares methods under generator-token compute normalization using multiple model and reward-model configurations, with a common large language model judge validated against native evaluations.

  • Benchmarks: Five benchmarks span medicine, law, finance, general chat, and creative writing, and none supports exact-match or binary verification.Evaluation therefore targets open-ended generation rather than tasks with straightforward exact verification.
  • TTS Methods: Five TTS families are represented by Best-of-N, Beam Search, Particle Filter, Sequential Refinement, Budget Forcing, and Fusion.Tree-search variants distinguish deterministic pruning from stochastic diversity, while Fusion synthesizes N candidates.
  • Models: Generators include OLMo3-7B-Think, OLMo3.1-32B-Think, and Qwen3.5 models at 9B and 35B-A3B scales.BoN uses Skywork-Reward-V2-Llama-3.1-8B and Llama-3.1-70B-Instruct-RM-RB2; tree search uses VersaPRM-8B.
  • Compute Normalisation: 23,800 GPU-hours were used overall, with compute normalized by total generator output tokens and four levels matched to average BoN output at N ∈{2, 4, 8, 16}.Discriminative reward-model cost is excluded because it is negligible relative to autoregressive decoding.
  • Evaluation: Qwen3.5-397B-A17B serves as the judge across benchmarks, with agreement validated on stratified 5–15% samples against each benchmark’s native judge.On HealthBench, the judge achieves Macro F1 0.679 with human annotations; on PRBench, criterion-level κ = 0.679.

5 Results

Across open-ended benchmarks, increasing compute improves the candidate pool but leaves exploitation largely stagnant: selection and search fail to identify better candidates. Fusion is the sole consistently scaling method, while refinement gains are benchmark-dependent and confounded.

  • Overall results: Oracle quality rises steadily with compute on every benchmark, but no selection or search method reliably identifies the better candidates.This establishes exploitation, rather than candidate generation, as the central bottleneck.
  • Overall results: BoN with either ORM yields negligible gains across an 8× budget increase, while tree search never beats the single-sample baseline and often degrades.Budget Forcing is flat or regresses even at low compute because these reasoning models exhaust useful thinking within modest token budgets.
  • Reward-model selection: Skywork-Reward-V2 and Llama-3.1-70B-RM produce near-identical realised scores across all benchmarks, both reaching 0.584 overall at XHigh.Their matching performance points to a structural rather than model-specific failure.
  • Fusion: 0.610 overall at XHigh is Fusion’s result, improving on every benchmark versus 0.584 for the best RM-based method and 0.574 baseline.Fusion is the only method that consistently scales, and this pattern holds across model scales.
  • Sequential Refinement: Sequential Refinement gains +7.3pp on WritingBench and beats Fusion on PRBench (0.342 vs. 0.331), but regresses −2.3pp on HealthBench and −4.5pp on LEXam.WritingBench gains are confounded by verbosity, while WildBench gains are driven almost entirely by a single subtask.

6 Analysis … A Compute Band Configurations

Across open-ended tasks, test-time scaling is limited by exploitation rather than exploration: strong candidate pools are generated, but selection often fails, while synthesis is the only consistently improving strategy. This gap persists across model families and scales, motivating better verifiers and exploitation mechanisms.

  • 6 Analysis: At high compute, candidate pools have high oracle quality, but exploitation remains the bottleneck: Fusion captures ∼40% of available headroom, while Particle Filter averages around −40%.Fusion and BoN produce nearly identical oracle pools, yet their realised scores diverge; Sequential Refinement ranges from −0.86 to 0.70 across benchmarks.
  • 6.1 Verifier Correlation Explains Selection Failure: 0.12: open-ended reward-model correlations with judge scores average 0.12 for Skywork and 0.11 for Llama, reducing BoN to near-random selection.On MATH-500 and GPQA Diamond, the generators gain only ∼2pp over single-sample baselines; verifier quality quantitatively predicts BoN headroom capture.
  • 6.2 Tree Search Failure: Diversity Collapse: 0.036–0.069: Particle Filtering has lower final-output cosine distances than BoN’s 0.123–0.124, collapsing nominally diverse particles into near-identical trajectories.On WritingBench, similarity remains ≥0.997 across 16 particles, so tree search fails in both exploration and exploitation.
  • 6.3 Generative Methods: Fusion Synthesises, Sequential Refinement Regresses: Fusion matches or exceeds BoN’s oracle ceiling despite generating fewer independent candidates, and its fused output consistently beats a held-out i.i.d. candidate.Synthesis can lift the pool ceiling more effectively than another independent sample at equal compute.
  • 6.3 Generative Methods: Fusion Synthesises, Sequential Refinement Regresses: Three of five benchmarks show Sequential Refinement below BoN’s oracle ceiling, while only PRBench exhibits genuine improvement across iterations and subtasks.WildBench’s aggregate gain traces to Coding & Debugging; excluding that subtask produces a net regression, while HealthBench and LEXam are counterproductive.
  • 6.3 Generative Methods: Fusion Synthesises, Sequential Refinement Regresses: +0.33: WritingBench’s within-prompt length–score correlation is +0.33 for Sequential Refinement versus +0.20 for BoN, confounding its apparent improvement.WritingBench’s judge discriminates more on length relative to substance, with content-to-length discrimination of 0.7× versus 4.3× on PRBench and 2.5× on HealthBench.
  • Appendix P.: ∼36%: Qwen3.5 models close ∼36% of headroom under Fusion, whereas OLMo3.1-32B-Think produces negative headroom on two of three benchmarks despite comparable or larger candidate-pool gaps.Within Qwen3.5, Fusion improves over single-sample baselines at every tested scale, but its advantage over a random candidate does not grow with model size.
  • 7 Conclusion: The study’s conclusion is that exploitation—not exploration—is the bottleneck: oracle quality rises with compute, but Fusion captures only ∼40% of available headroom and Sequential Refinement yields genuine gains on only one benchmark.The paper proposes verifier calibration using ρ̂_v and exploitation mechanisms beyond selection and synthesis; it also cautions that conclusions rely on two model families, one unified judge, and an approximate oracle estimator for several methods.

B Benchmark Details · C Hyperparameters and Prompt Templates

The evaluation spans five open-ended benchmarks with distinct datasets, rubric structures, and single- or multi-turn formats. Across methods, generation uses standardized sampling and system prompts, while algorithm-specific settings and custom templates define refinement and fusion behavior.

  • B Benchmark Details: LEXam retains 516 English open-question items from a 2,841-item release, while WritingBench retains 555 English items from 1,000 bilingual items.WritingBench uses five dynamic criteria per item, each scored from 1–10, with the example score equal to their mean.
  • B Benchmark Details: HealthBench contains 5,000 examples partitioned into seven non-overlapping theme-tagged evaluation tasks.The task sizes range from 360 to 1,097 examples, and each example carries an average of 11.4 physician-authored rubric criteria.
  • B Benchmark Details: PRBench comprises four configurations totaling 1,650 items, each evaluated with 10–30 weighted rubric criteria.The dataset contains 29,252 criteria, averaging approximately 17 per item, with criterion-specific integer weights.
  • B Benchmark Details: HealthBench, PRBench, and WildBench include multi-turn items, whereas LEXam and WritingBench are single-turn only.Multi-turn conversations are passed to generators and judges as ordered role-content message lists.
  • C Hyperparameters and Prompt Templates: All generation uses generator-family sampling profiles with seed 42 and the shared system prompt “Please complete the following user request.”Fusion synthesis and Budget Forcing reduce max_tokens from 16,384 to 8,192.
  • C Hyperparameters and Prompt Templates: BoN generates N completions and returns the ORM-scored argmax, while Beam Search and Particle Filter use PRM-based path scoring and resampling.BoN lower-band results bootstrap subsamples from a shared N=16 XHigh pool; Particle Filter uses softmax(w/T) with T=1.0.
  • C Hyperparameters and Prompt Templates: Sequential Refinement iteratively critiques and revises one draft using a maximum context window of J=3 previous iterations.Each iteration uses feedback, stop-check, and refinement calls, with forced iteration making the stop decision unused.
  • C Hyperparameters and Prompt Templates: Fusion synthesizes a shuffled random subset of independently generated candidates, while BoN, Beam Search, Particle Filter, and Budget Forcing use only native and system prompts.Fusion uses subset sizes s ∈{3, 7, 15} and the direct-synthesis variant.

D Judge Agreement Analysis … Heterogeneous attenuation across benchmarks.

Judge quality varies substantially across benchmarks, while theory explains how verifier correlation, judge noise, and candidate-pool structure govern test-time scaling estimates. The unified judge agrees well with physicians on HealthBench, but agreement attenuates on heterogeneous benchmarks and measured relationships remain predictable in aggregate.

  • D Judge Agreement Analysis: 0.679 Macro F1: Qwen3.5-397B-A17B matches physician annotations comparably to GPT-4.1 and o4-mini, exceeding the majority physician’s pairwise agreement.Physician agreement averaged 0.655 Macro F1, placing the unified judge at the 57th percentile among 184 physicians.
  • D Judge Agreement Analysis: QWK = 0.564 and 0.408: unified-versus-native agreement is strongest for PRBench and LEXam but lower for WildBench and WritingBench.The lower agreement is attributed to heterogeneous evaluation criteria across task types and domains.
  • E Theory: ρv = 1 captures all available quality, ρv = 0 makes additional compute ineffective, and ρv < 0 makes additional compute harmful for BoN selection.Self-verifier methods instead depend on the generator’s capability to produce and assess high-quality outputs, not parameter count alone.
  • F Oracle Bias From Judge Noise: ≤0.20 survival probability: at ϵprune = 0.33 and d = 4, tree search returns a suboptimal candidate more than 80% of the time.This formalizes why Beam Search and Particle Filter can underperform naive parallel sampling when PRM miscalibration is severe.
  • F Oracle Bias From Judge Noise: Judge noise inflates the naive oracle’s maximum, whereas the ideal oracle is the true quality of the genuinely best candidate.The naive oracle is O_naive = max_i X_i; the ideal oracle is O_ideal = max_i μ_i.
  • F.1 Derivation of the expectations: a_n grows with n: the naive-to-ideal oracle gap vanishes as σJ → 0 or τ → ∞, equals a_nσJ when τ = 0, and increases with n.These behaviors follow from the Gaussian candidate-quality and judge-noise model.
  • Ideal per-entry estimator: The plug-in oracle correction is conservative for Fusion and Refinement but can be a slight upper bound for Particle Filter because candidate-pool structure violates the i.i.d. assumption.The closed-form correction is exact only for i.i.d. Gaussian candidates, so residual error depends on each algorithm’s structural deviations.
  • Heterogeneous attenuation across benchmarks.: 1.198 slope and R2 = 0.656: heterogeneous τ and σJ create benchmark-level scatter but do not bias the measured h ≈ρv relationship in expectation.Judge noise attenuates measured quantities symmetrically, so oracle correction is unnecessary for Equation 4’s validity but remains useful for unbiased true-oracle estimates.

G Naive Oracle Results … H.3 Qwen3.5-9B Oracle Results

Across naive-oracle analyses and Qwen3.5 results, oracle quality increases with compute while realised quality stagnates, leaving exploitation as the binding constraint. This conclusion holds across oracle conventions, with headroom capture bounded by verifier correlation.

  • G Naive Oracle Results: Oracle quality rises with compute while realised quality stagnates, so exploitation remains the binding constraint despite the naive oracle’s judge-noise inflation.The naive oracle overstates true pool quality and makes the exploitation gap appear more severe, not less.
  • G Naive Oracle Results: The naive-oracle figures compare oracle and realised quality at matched XHigh compute for Qwen3.5-35B-A3B.Figure 7 uses full bars for oracle quality and hatched bars for realised quality.
  • G.1 Headroom capture tables (Unadjusted oracles): Headroom capture at XHigh is evaluated as exploitation quality for Qwen3.5-35B-A3B under the naive oracle.The naive-oracle headroom-capture analysis is explicitly compared with the bias-corrected analysis.
  • G Naive Oracle Results: Headroom capture is bounded by verifier correlation regardless of whether the oracle and correlation estimate are denoised.Across 152 task-level points, the naive-oracle fit closely tracks y = x, mirroring the bias-corrected relationship ĥ ≈ ρ̂v.
  • G.1 Headroom capture tables (Unadjusted oracles): Headroom capture h measures the fraction of the oracle gap recovered relative to the BoN N=1 baseline: h = 1 is full capture, h = 0 matches baseline, and h < 0 underperforms it.The same definition is reported for Qwen3.5-35B, OLMo3-32B, Qwen3.5-9B, and OLMo3-7B.
  • H.1 Qwen3.5-35B-A3B Oracle Results: Qwen3.5-35B-A3B oracle quality is reported across five benchmarks and four compute levels, with gains measured against the BoN oracle N=1 baseline.Cells distinguish large and small gains, with bold marking gains ≥0.03 over baseline.
  • H.2 Qwen3.5-9B Realised Results: Qwen3.5-9B realised quality is reported across five benchmarks and four compute levels relative to the single-sample baseline, BoN@1.Green denotes improvement, red regression, and bold marks gains ≥0.03 over baseline.
  • H.3 Qwen3.5-9B Oracle Results: Qwen3.5-9B oracle quality is reported across five benchmarks and four compute levels against the BoN oracle N=1 baseline.The table marks large and small gains by cell colour and bolds gains ≥0.03 over baseline.

I OLMo3 Results … K Verifier Correlation and Headroom Capture by RM

OLMo3 shows that generation capability does not imply exploitation capability: candidate quality rises with compute, while selection and synthesis can degrade realised quality. Across verifier analyses, headroom capture closely tracks verifier correlation, linking poor exploitation to weak reward-model quality.

  • I OLMo3 Results: OLMo3 is the primary negative case: generation capability does not imply exploitation capability.The results directly support the paper’s central thesis about the dissociation between generating candidates and exploiting them.
  • I OLMo3 Results: OLMo3-32B Fusion falls from 0.475 baseline to 0.441 at Mid and 0.394 at High, with regressions on four benchmarks and only marginal improvement on LEXam.On PRBench, realised quality drops from 0.211 baseline to 0.164 at High (−22% relative); the damage worsens with additional compute.
  • I OLMo3 Results: OLMo3-7B Fusion improves at High on HealthBench (+3.0pp), LEXam (+2.3pp), PRBench (+1.2pp), and WritingBench (+1.2pp), but regresses on WildBench at Mid (−1.7pp).This partial success suggests the OLMo3-32B failure is not a simple architectural property, but depends on synthesis capability and scaling.
  • I.1 Oracle Results Olmo-7B: OLMo3-7B oracle quality rises from 0.356 at N=1 to 0.474 at XHigh, with Fusion oracle closely tracking BoN oracle across compute levels.The candidate pool is proportionally comparable to Qwen3.5, indicating that realised-quality divergence is driven by exploitation rather than exploration.
  • I.2 Oracle results Olmo3-32B: OLMo3-32B oracle quality is reported separately from realised quality, enabling the same exploration–exploitation comparison for the larger model.Table 25 defines oracle quality across five benchmarks and four compute levels relative to the BoN oracle N=1 baseline.
  • J Headroom capture tables: Headroom capture is defined as h = (realized −µ)/(oracleideal −µ), where h = 1 means full exploitation, h = 0 matches the BoN N=1 baseline, and h < 0 underperforms it.The tables apply this definition to Qwen3.5-35B-A3B, Qwen3.5-9B, OLMo3-32B, and OLMo3-7B.
  • J.1 Qwen3.5-35B-A3B results: The headroom-capture tables compare Qwen3.5-35B-A3B and Qwen3.5-9B against their BoN N=1 baselines using the same oracle-gap metric.Positive values indicate captured gains, negative values indicate performance below baseline, and bold marks |h| ≥ 0.20.
  • K Verifier Correlation and Headroom Capture by RM: Across 152 generator–reward-model–benchmark combinations, headroom capture tracks verifier correlation: h = 1.198ρv −0.011, R2 = 0.66, ρ = 0.81, p < 10−36.The same relationship is reported as h = 1.20ρv −0.011 in the headroom table, validating Equation 4.

L Deterministic Benchmark Results: MATH-500 and GPQA Diamond

On deterministic benchmarks with binary verification, Qwen3.5-35B-A3B starts near the oracle on MATH-500 and GPQA Diamond, leaving limited headroom. RM-based Best-of-N captures substantially more headroom here than on open-ended QA, especially on MATH-500.

  • Benchmark setting: MATH-500 and GPQA Diamond provide exact-match or multiple-choice verification, making them canonical settings for test-time scaling.The benchmarks cover competition mathematics and graduate-level science, respectively.
  • Evaluation setup: The evaluation uses Qwen3.5-35B-A3B, identical TTS methods, and matched token-budget normalisation for direct cross-setting comparison.This setup supports comparing method rankings and headroom capture between deterministic and open-ended tasks.
  • Results: 0.922 and 0.823 are Qwen3.5-35B-A3B’s single-sample pass@1 scores on MATH-500 and GPQA Diamond, respectively.These leave 3.2 pp and 13.7 pp of headroom to oracle Pass@16.
  • Results: 50% and 15% are the fractions of available headroom captured by RM-based BoN at XHigh on MATH-500 and GPQA, respectively.The corresponding figure reported for open-ended QA is ∼15%.

M Skywork-Reward-V2 Reproduction on Deterministic Benchmarks … O.2 Native WritingBench Judge: Bias Amplification

Across deterministic and open-ended benchmarks, additional candidates improve oracle quality, but exploitation remains unreliable: refinement gains are concentrated, verbosity can distort scores, and native judging amplifies this bias on WritingBench.

  • M Skywork-Reward-V2 Reproduction on Deterministic Benchmarks: At N=32, realised accuracy reaches ∼0.66 on MATH-500 and ∼0.55 on GPQA, reproducing Skywork-Reward-V2’s reported positive BoN scaling.The oracle approaches 1.0 while realised quality plateaus below 0.7, although correctness remains positively correlated with reward-model scores.
  • N WildBench Per-Category Decomposition: Coding & Debugging comprises 16.6% of WildBench items but contributes 105.5% of total realised refinement gains.Its mean lift is +27.06 per item, versus −0.28 across the remaining 854 items; excluding it, refinement produces a net regression.
  • N WildBench Per-Category Decomposition: Without Coding & Debugging, WildBench quality regresses across refinement iterations while the oracle continues to rise.This pattern confirms exploitation failure outside the near-deterministic coding subtask.
  • O.1 Unified Judge: Length–Score Correlation: Sequential Refinement increases response length monotonically across iterations, so apparent WritingBench gains can reflect elaboration rather than improved content.BoN and Fusion maintain stable length distributions, whereas refinement shows the steepest growth on WritingBench and PRBench.
  • O.1 Unified Judge: Length–Score Correlation: WritingBench has the strongest unified-judge length–score correlations: +0.198 for BoN and +0.352 for Sequential Refinement.For refinement, the CVscore/CVlen ratio is 0.36, indicating length growth is the primary driver of score gains.
  • O.1 Unified Judge: Length–Score Correlation: HealthBench and PRBench are content-driven, with CVscore/CVlen ratios of 1.57 and 3.17 under BoN.Their refinement regressions therefore reflect genuine quality degradation rather than scoring artifacts.
  • O.1 Unified Judge: Length–Score Correlation: LEXam is the only benchmark with a negative refinement length–score correlation, ˆρ = −0.060, p < 10−3.The result is consistent with longer but worse responses on knowledge-intensive legal tasks.
  • O.2 Native WritingBench Judge: Bias Amplification: The native WritingBench judge raises length–score correlation to +0.370 for BoN and +0.539 for Sequential Refinement.WritingBench CVscore falls to 0.034 for BoN and 0.038 for refinement, while other benchmarks remain unchanged, indicating benchmark-intrinsic verbosity bias.

O.3 WritingBench: Judge Robustness Analysis · P Effect of Model Capacity on Self-Verifier Algorithms

Judge robustness does not alter the main WritingBench conclusions: BoN transfers across judges, while Sequential Refinement’s apparent improvement is explained by denominator compression from length bias. Across model capacities, Fusion scales reliably for Qwen3.5, whereas Sequential Refinement remains benchmark-dependent and its exploration advantage is generally weak.

  • O.3 WritingBench: Judge Robustness Analysis: 0.19 vs. 0.19 headroom capture for BoN with either Skywork or Llama70B is judge-invariant, supporting clean transfer of parallel selection results.The comparison uses the same TTS outputs rescored by the official WritingBench Critic model.
  • O.3 WritingBench: Judge Robustness Analysis: Critic-model length bias pulls late, verbose Refinement iterations’ realized and oracle scores together, mechanically inflating capture without genuine selection improvement.This explains the judge divergence while leaving methods-level conclusions stable where it matters.
  • P Effect of Model Capacity on Self-Verifier Algorithms: Both absolute score gain and relative error reduction are used to assess model-capacity effects against the BoN@1 baseline.Relative error reduction measures the fraction of the gap to a perfect score that is closed.
  • P Effect of Model Capacity on Self-Verifier Algorithms: Qwen3.5 Fusion consistently helps across benchmarks with roughly scale-stable gains, while additional compute improves performance monotonically.Olmo3.1-32b fails badly, and Olmo3-7b still struggles to reach the BoN@1 baseline.
  • P Effect of Model Capacity on Self-Verifier Algorithms: Sequential Refinement gains on 2/5 benchmarks but strictly hurts on 2/5, with no consistent model-capacity scaling trend.Its performance therefore depends strongly on the benchmark rather than increasing predictably with model size.
  • P Effect of Model Capacity on Self-Verifier Algorithms: BoN’s oracle beats Sequential Refinement on 3/5 matched-compute benchmarks, ties on WildBench, and loses on WritingBench under a known length bias.The WritingBench exception favors Refinement because the judge rewards its length-growing trajectory.
  • P Effect of Model Capacity on Self-Verifier Algorithms: At matched candidate counts, self-revisions show strictly negative effects on two benchmarks, small effects on two, and a large positive effect only on WritingBench.The WritingBench result is attributed to judge length bias combined with Refinement’s increased verbosity.

Q Task Demand Profiles · R Token usage

Benchmark demand profiles describe which capabilities tasks stress, but method success depends on how those demands are operationalised and evaluated. The token-usage tables document compute-normalised completion-token budgets across TTS methods and scaling levels.

  • Q Task Demand Profiles: Demand profiles annotate five benchmarks across 18 dimensions spanning primordial capabilities, domain knowledge, and extraneous factors.Radar plots encode demand scores from 0 to 5 and instance frequency through colour intensity.
  • Q Task Demand Profiles: Demand profiles characterise benchmark demands but do not predict TTS benefits, because exploitation bottlenecks mediate whether methods engage those skills effectively.The mapping is interpretive rather than predictive.
  • Q Task Demand Profiles: HealthBench is dominated by Metacognition & Critical Thinking and Comprehension & Expression, while Knowledge and Quantitative & Logical demands are comparatively absent.Its low knowledge load reflects reasoning over medical queries and structured responses rather than raw factual recall; Sequential Refinement regresses −2.3pp at XHigh.
  • Q Task Demand Profiles: LEXam is dominated by Comprehension & Expression and Attention & Scan, reflecting information integration across legal texts and interpretive legal reasoning.Sequential Refinement shows the strongest regression across benchmarks, at −4.5pp at XHigh.
  • Q Task Demand Profiles: The same dominant dimensions can yield opposite Sequential Refinement outcomes: PRBench gains +3.5pp at XHigh, whereas HealthBench regresses −2.3pp.The difference is attributed to evaluation operationalisation: PRBench rewards elaboration and structured reasoning, while HealthBench penalises deviation from precise clinical content.
  • Q Task Demand Profiles: WildBench’s MC+CE profile resembles HealthBench, yet Sequential Refinement gains +2.4pp at XHigh because its holistic checklist judge tolerates elaboration more than physician rubrics.WritingBench’s CE+CL profile plausibly supports refinement, but its +7.3pp gain is substantially confounded by judge verbosity bias.
  • Q Task Demand Profiles: Demand dimensions alone do not determine method success; precision-sensitive judges penalise elaboration, whereas checklist and holistic judges tolerate or reward it.This evaluation-operationalisation interaction is distinct from model capability.
  • R Token usage: Token-usage accounting reports total completion tokens for BoN, Particle Filter, Beam Search, Sequential Refinement, and Fusion across Mid, High, and XHigh settings.The tables include a ±27% compute band for BoN and specify method parameters such as n, beam width, iterations, and synthesis count.
Loading 2608.18931v1…