Source-linked AI summary
More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix It
Haohui Yang, Jiaxing Sun, Xiujun Ma
TL;DR
Power Sampling can increase probability mass on correct trajectories yet worsen multi-sample decisions because fixed sharpening creates dose and coverage mismatches. The paper introduces Relative-Rank SoftSat and weighted aggregation over a shared candidate pool, while finding that global Power underperforms uniform consensus across all six LiveAoPSBench and PHYSICS settings.
Problem
Power Sampling can increase correct-trajectory mass while weakening the answer-level support needed for aggregation, so trajectory sharpening may worsen final multi-sample decisions.
Method
Relative-Rank SoftSat calibrates sharpening with within-prompt relative rank, saturates gains on dominant trajectories, and supports weighted aggregation over a shared Base candidate pool.
Results
Power underperforms Uniform in all six LiveAoPSBench and PHYSICS settings, by 1.808–18.474 points on LiveAoPSBench and 1.934–6.999 points on PHYSICS.
Takeaways & Limitations
Reliable multi-sample inference requires controlling both sharpening dose across problems and coverage of moderate-probability reasoning paths.
Takeaways & Limitations
The asymptotic weighted-consensus result applies to an ideal target and does not account for finite-M Power-SMC approximation error or produce fresh target samples.
Abstract
from arXiv · showhide
Power Sampling sharpens a language model's distribution over complete generation trajectories, offering a verifier-free way to improve reasoning at inference time. It also has the potential to serve as a general-purpose front end for a broad range of downstream sampling methods. However, we uncover a striking paradox: Power Sampling can drive more probability mass toward correct trajectories while degrading the downstream inference it is intended to enhance. Using self-consistency as a representative case, we observe accuracy drops of up to 18.5 percentage points across models and reasoning benchmarks. We trace this paradox to two mismatches. Dose mismatch arises because a fixed exponent induces drastically different amounts of distributional change across problems. Coverage mismatch arises because global sharpening concentrates mass on a narrow set of dominant paths: high pass@k, often interpreted as evidence of preserved diversity, can therefore coexist with the loss of broad reasoning-path support required for downstream aggregation, search, and selection. Guided by this diagnosis, we replace uniform trajectory exponentiation with a deformation-controlled, support-preserving Power target that calibrates sharpening across problems while limiting the suppression of moderate-probability paths. In a same-budget instantiation with weighted self-consistency, the repaired sampler reverses the losses caused by global Power and outperforms standard multi-sample inference across reasoning benchmarks.
1 INTRODUCTION
Power Sampling can increase probability mass on individually correct trajectories while degrading multi-trajectory aggregation because global sharpening creates coverage and dose mismatches. The paper introduces Relative-Rank SoftSat and a shared-pool importance-weighted realization to control sharpening and preserve target decisions.
- Introduction: Seven of nine model–benchmark settings degrade when Base SC8 is replaced by Power with α = 4.This demonstrates that sharper trajectory distributions can conflict with multi-trajectory aggregation despite favoring individually likely paths.
- Introduction: Relative-Rank SoftSat replaces absolute likelihood with within-prompt rank and saturates gains on dominant trajectories to control dose and preserve coverage.It promotes moderately ranked trajectories while limiting disproportionate mass absorption by a few dominant paths.
- Introduction: A shared Base candidate pool enables importance-weighted aggregation to realize target sampling with the same N trajectories as Base+SCN.The analysis shows this estimator asymptotically preserves direct target sampling’s decision under a standard positive-margin condition.
- Introduction: Power Sampling can increase correct-trajectory mass yet degrade multi-trajectory decisions through coverage mismatch and dose mismatch.Coverage mismatch concerns collective answer-level support, while dose mismatch concerns inconsistent sharpening strength across prompts.
2 RELATED WORK
Related work frames Power Sampling as inference-time distribution sharpening via trajectory-probability exponentiation, while existing evaluations emphasize single-output accuracy and pass@k. Multi-sample aggregation methods differ in how they define agreement and weight candidate support.
- Distribution sharpening: Distribution sharpening reallocates probability toward outputs favored by a model, and Power Sampling approximates trajectory exponentiation using autoregressive Markov chain Monte Carlo.Iterative MCMC introduces substantial sequential latency.
- Evaluation limitations: Existing evaluations mainly use single-output accuracy and pass@k, although pass@k only indicates whether a correct trajectory remains reachable.It does not characterize probability allocation across reasoning paths or answer modes.
- Multi-sample aggregation: Multi-sample aggregation methods vary in how they define agreement and how much support each candidate contributes.These two design choices organize the surveyed aggregation approaches.
- Agreement definitions: Self-Consistency groups trajectories by normalized final answer and selects the largest group, while CodeT evaluates generated programs and tests through execution.Universal Self-Consistency uses an LLM to identify the most consistent response when no canonical short answer exists.
- Support weighting: Standard Self-Consistency gives each trajectory one vote, whereas CISC, RASC, and RPC weight support using confidence, rationale quality, answer consistency, frequency, or sequence probability.These methods represent different support-weighting choices for candidate answers.
3 PRELIMINARIES
This section defines trajectory probabilities, distinguishes token-level temperature sampling from sequence-level Power Sampling, and introduces the answer-mass quantities used to analyze downstream aggregation. It also formalizes Self-Consistency and ModeX as agreement-based readouts over multiple sampled trajectories.
- Trajectory distributions: A complete generation trajectory τ = (τ1, . . . , τT) receives a probability and log-likelihood from the base model.Prompt conditioning is omitted for clarity.
- Trajectory distributions: Token-level temperature sampling reshapes each next-token distribution separately, whereas Power Sampling sharpens the distribution over complete trajectories.For Power Sampling, the ideal target uses exponent α > 1.
- Trajectory distributions: Token-temperature and Power targets generally differ because token-level normalization depends on the generated prefix; Power-SMC approximates the sequence-level target with M particles.The analysis treats ideal qα as the reference and separates finite-M approximation error.
- Downstream readouts: A trajectory distribution µ induces answer mass and total correct-trajectory mass, while Self-Consistency selects an answer from N independently sampled trajectories.With a unique gold answer a⋆, correct-answer mass satisfies Cµ = Qµ(a⋆).
- Downstream readouts: ModeX derives consensus from a similarity graph for executable or open-ended outputs, so both readouts convert support across multiple trajectories into a final decision.Self-Consistency and ModeX use different notions of agreement.
4 MISMATCH ANALYSIS
The analysis identifies two distinct mismatches behind Power Sampling’s failures: coverage mismatch concentrates mass within and across competing answers, while dose mismatch makes a fixed exponent deform different problems unequally. Thus, higher total correct mass or pass@k does not guarantee sufficient support or margins for downstream aggregation.
- Coverage mismatch: High pass@k can coexist with a smaller answer margin because it measures whether any correct trajectory appears, not how correct and incorrect mass are organized.Pass@k depends only on total correct mass and does not characterize answer competition or the distribution of mass among trajectories.
- Coverage mismatch: Power increases total correct mass while reducing the correct answer’s margin, as the strongest incorrect answer gains even more mass.This answer-level shift does not reveal how the added correct mass is distributed within the correct set.
- Coverage mismatch: Power concentrates correct mass into a narrower, higher-likelihood subset, showing that total correct mass alone is insufficient to characterize support for aggregation.Coverage also concerns concentration within the correct set, distinct from competition between answer-level masses.
- Dose mismatch: A shared α imposes unequal deformation across problems because narrow likelihood gaps yield mild reweighting while wide gaps sharply concentrate distributions.Power acts on raw sequence-level likelihood rather than a problem-relative scale, so the same exponent does not produce comparable changes.
- Dose mismatch: Figure 2(c) shows fixed-α curves fanning outward, with a small set of problems receiving markedly larger gains.Problem-specific likelihood dispersion controls the leading-order movement of the entire Power target.
5 METHOD
The method replaces raw likelihood-based Power weighting with relative-rank SoftSat shaping to calibrate dose across problems while preserving useful coverage. It applies the resulting weights to consensus readouts using one shared pool of Base trajectories, without additional model calls.
- Single-Pool Readout: One pool of N Base trajectories estimates the answer mass of the ideal Power target, avoiding N independent Power-SMC runs.The construction uses importance weighting rather than repeatedly sampling from the target.
- Theoretical Scope: The asymptotic equivalence result applies only at the ideal-target level and does not remove finite-M Power-SMC error or create fresh target samples.With a finite answer space and a uniquely separated mode, both estimators select the same mode with probability tending to one.
- Relative-Rank SoftSat: Relative-rank SoftSat replaces raw likelihood gaps with within-problem ranks and saturates gains for high ranks, addressing dose and coverage mismatch.The repaired target uses qβ,m(τ) ∝ p(τ)rβ,m(τ), with SoftSat increasing gains for moderate ranks and saturating them for u ≥ m.
- Consensus Readouts: The weighted candidates feed task-appropriate consensus readouts, including weighted Self-Consistency for answer-based tasks and ModeX for free-form responses.Uniform weights recover the original readout in either case.
- Implementation: The method reuses the same N Base trajectories as uniform consensus and incurs the same model-generation cost with no additional model calls.Candidate weights are obtained through ranking, SoftSat shaping, bounded mean-one rescaling, and normalization.
6 EXPERIMENTS
Experiments across code, mathematics, and physics show that Power’s effect varies by response space and model, with substantial regressions relative to Uniform. SoftSat removes Power’s large failures while preserving gains, supports stronger sharpening at larger budgets, and preserves useful coverage in diagnostic and finite-sample checks.
- Main results: Power underperforms Uniform in all six LiveAoPSBench and PHYSICS settings, with regressions of 1.808–18.474 and 1.934–6.999 points, respectively.On BigCodeBench, Power improves Qwen3.5 and Ministral but degrades Nemotron, demonstrating variation across response spaces and models.
- Main results: SoftSat improves five of nine settings, ties one, and limits its largest regression to 1.004 points relative to Uniform.For Nemotron’s two largest Power failures, gaps fall from 18.474 to 1.004 points on LiveAoPSBench and from 6.999 to 0.255 points on PHYSICS.
- Budget sweep: The best SoftSat β rises from 0.25 at N = 4 to 2 at N = 32, while SoftSat improves over uniform SC at every evaluated budget.A larger candidate pool supports stronger reweighting, but accuracy remains benchmarked against the corresponding uniform-consensus baseline.
- Coverage diagnostics: Power shifts the answer-level center from (C, G) = (0.433, 0.132) to (0.464, −0.072), whereas SoftSat gives (0.439, 0.133) and retains coverage close to Base.Figure 4 reports that SoftSat restores the gold-answer margin without discarding correct-answer mass and avoids Power’s narrow concentration within the correct set.
- Weighted realization: On 1,118 human-verified MoreHopQA Case-5 examples, repeated Power-SMC and its weighted realization select the same normalized answer in 99.55% of cases.Their accuracies differ by 0.089 percentage points, with a paired-bootstrap 95% confidence interval of [−0.268, 0.447], while mean inference time falls by 7.42×.
7 CONCLUSION
The paper explains why sequence-level Power Sampling can increase probability on correct trajectories while worsening decisions based on multiple samples. It attributes this failure to coverage mismatch and dose mismatch, motivating Relative-Rank SoftSat.
- Power Sampling can improve the probability of correct trajectories yet degrade decisions formed from multiple samples.The conclusion frames this as the central failure studied in the paper.
- Coverage mismatch concentrates probability mass on a narrow set of dominant trajectories and answers.
- Dose mismatch makes a fixed exponent induce sharply different deformations across problems.
- This diagnosis led to Relative-Rank SoftSat, which replaces absolute likelihood with a problem-rel...The supplied passage truncates the method description after “problem-rel”.
A FULL MISMATCH DIAGNOSTICS
The section broadens the paper’s mismatch diagnostics across all evaluated models on LiveAoPSBench, while extending the coverage comparison in parallel.
- Figure 5 extends the three diagnostics from Figure 2 to every evaluated model on LiveAoPSBench.
- Figure 6 extends the coverage comparison from Figure 4.
B BUDGET SWEEP ACROSS BENCHMARKS
Figure 7 extends the budget-sweep analysis from Figure 3 to all three evaluated benchmarks.
- Figure 7 extends Figure 3 to the three evaluated benchmarks.
C FINITE-SAMPLE CHECK OF THE WEIGHTED REALIZATION
A finite-sample check on 1,118 human-verified MoreHopQA Case-5 examples finds that weighted Power-SMC aggregation closely matches repeated sampling while slightly improving accuracy. Most final decisions are unchanged, and the answer distributions remain highly similar.
- Experimental setup: The check evaluates Qwen3.5-9B with α = 4 on all 1,118 human-verified MoreHopQA Case-5 examples.The sampling reference uses an M = 8 particle Power-SMC system independently N = 8 times.
- Experimental setup: The reference applies uniform self-consistency to eight returned answers, whereas weighted realization aggregates normalized answer mass from one Power-SMC run with M = 8 terminal particles.This compares repeated sampling against a single weighted Power-SMC realization under the same particle count.
- Results: 1,113 of 1,118 final decisions are unchanged; weighting corrects three errors and introduces two, producing 91.14% accuracy versus 91.06% for repeated sampling.The difference is 0.089 percentage points, with a paired-bootstrap 95% confidence interval of [−0.268, 0.447].
- Results: 80.86% of answer distributions have DTV at most 10−7, 94.36% have DTV at most 0.01, and only 0.36% exceed 10−0.5.These figures indicate near agreement between the weighted and repeated-sampling answer distributions.
D PROOFS FOR WEIGHTED TARGET APPROXIMATION
The proofs establish that the weighted target estimator is unbiased and converges to the target answer distribution, with finite-sample error controlled by its variance. When the target has a unique mode with positive margin, both estimators select that mode with probability tending to one.
- Consistency: Both self-normalized estimators converge almost surely to Qqα(a).The numerator and denominator converge almost surely, and bounded convergence identifies the ratio with the target answer mass.
- Estimator properties: The target-sampling estimator is unbiased for every answer and sample size N.For target samples, the answer indicator is Bernoulli with mean Qqα(a), yielding E[bQsample_N(a)] = Qqα(a).
- Estimator properties: The target-sampling estimator has variance θ(1 − θ)/N, and its error relative to the Base-pool estimator is controlled by a second-moment bound.Independence makes the cross term between centered errors vanish; the resulting bound implies an O(N^-1/2) rate.
- Mode agreement: With a finite answer space and unique target mode margin δ > 0, both estimators select the same mode with probability tending to one.A union bound ensures all estimated masses eventually lie within δ/3 of their targets, preserving the unique mode.