REWIND: Uncertainty-Guided Trajectory Repair for Test-Time Reasoning

1 Introduction

• Revision-site selection is a practical bottleneck because a random cut can preserve a mistaken premise or replace substantial correct work.Post-regeneration filtering can accept or reject a completed proposal but cannot guide where regeneration begins.Source: §1 · p. 1
• Incorrect responses have approximately 112% higher mean token entropy than correct responses in the MATH500 repeated-generation study.The study reports an average within-question relative increase.Source: §1 · p. 1
• REWIND selects suffix-regeneration sites by backtracking from a high-entropy peak to the largest preceding entropy increase and aligning the cut with a natural delimiter.An MH-style score filter then decides whether to retain the regenerated proposal.Source: §1 · pp. 1–2
• Controlled ablations support entropy-gradient backtracking and boundary alignment, while further evaluations report transfer to few-shot prompting and code generation and stability across random seeds.These findings are reported alongside the main benchmark comparison.Source: §1 · p. 2
• Across three model families and five benchmarks, REWIND achieves the highest accuracy among baselines, improving over random-proposal MCMC by 0.51–10.00 percentage points.The largest reported improvement is on AIME2025 with DeepSeek-R1-Distill-Qwen-1.5B, from 16.67% to 26.67%.Source: §1 · p. 2

2 Related Work

• Inference-time methods include self-consistency aggregation (Wang et al., 2023), branch exploration with Tree of Thoughts and MCTS (Yao et al., 2023; Zhou et al., 2023), and iterative refinement such as Self-Refine (Madaan et al., 2023).Self-Refine uses model-generated textual feedback, which can be unreliable without informative feedback (Huang et al., 2023).Source: §2 · p. 2
• Sampling-based refinement ranges from word-level Metropolis–Hastings edits (Miao et al., 2019) and block rewrites (Forristal et al., 2023) to Power Sampling suffix proposals (Karan and Du, 2025).REWIND shifts attention from how revisions are generated to where they are applied.Source: §2 · p. 2
• Uncertainty-guided methods use model signals for token-level decoding, response-level sample allocation, or search-level exploration decisions.Examples include DoLa and typical decoding (Chuang et al., 2024; Meister et al., 2023), adaptive consistency (Aggarwal et al., 2023), semantic entropy (Kuhn et al., 2023), and Entro-duction (Zhang et al., 2025).Source: §2 · p. 2
• REWIND uses local uncertainty to select a cut point for suffix regeneration within an existing response, without updating model parameters.This contrasts with using token uncertainty to focus RLVR policy-gradient updates (Wang et al., 2026) or to filter or terminate reasoning traces (Fu et al., 2026).Source: §2 · pp. 2–3

3 Preliminaries and Observations

• The section frames repeated-generation entropy analysis as an empirical study of a possible during-generation signal for detecting errors.The analysis computes token entropy from the full-vocabulary next-token distribution before sampling transformations.Source: §3 · p. 3, §3.2 · p. 3

3.1 Power Sampling for LLMs

• Power Sampling targets higher-likelihood complete responses, with an MH update whose acceptance rule cancels the unknown normalization constant.A random-suffix proposal chooses a cut point and regenerates the continuation; the later section specifies the state-dependent proposal and implemented MH-style filter.Source: §3.1 · p. 3

3.2 Empirical Entropy–Error Associations

• Incorrect responses have an approximately 112% mean per-question relative entropy increase, and mean token entropy distinguishes incorrect from correct responses with AUROC 0.746.Incorrect-response entropy is higher on 88.12% of paired questions; mean token entropy provides the strongest discrimination among the three response-level statistics.Source: §3.2 · pp. 3–4, Table 1 · p. 4
• Mean entropy of incorrect responses rises across question groups with increasing numbers of correct answers among 20 generations.The analysis averages the mean entropy of incorrect responses equally across questions in each success-count group.Source: §3.2 · p. 4, Figure 1 · p. 4
• For each of the 500 MATH500 questions, the study samples N = 20 responses from Qwen2.5-Math-7B; whether entropy identifies useful revision sites is tested directly through proposal ablations.Source: §3.2 · pp. 3–4

4 Method

• REWIND is an entropy-guided method for selecting where to revise a reasoning trajectory.The method uses signals from the generation process to propose revisions.Source: §4 · p. 4

4.1 Entropy-Guided Suffix Proposals

• A proposal selects the highest-entropy generated position, backtracks to the largest adjacent entropy increase, then aligns the cut to a preceding delimiter before regenerating the suffix.If no delimiter is found, the cut remains at the backtracked position.Source: §4.1 · pp. 4–5
• A shared-horizon, state-dependent MH-style filter accepts or rejects the regenerated suffix, but omits site-selection probabilities and does not establish exact MH stationarity.REWIND can add an entropy-guided proposal after Power Sampling's random-cut proposals, with all proposals using the same acceptance score.Source: §4.1 · p. 5

4.2 Theoretical Intuition

• The theoretical intuition decomposes entropy into within-interpretation uncertainty and cross-interpretation disagreement, whose sharp increase can produce a gradient spike before a later entropy peak.This motivates candidate selection but does not guarantee a correct retained prefix or causal error localization.Source: §4.2 · p. 5

5.1 Experimental Setup

• The evaluation covers three model families, five benchmarks, and decoding, revision, MCMC, and candidate-selection baselines.Table 2 reports accuracy averaged over three runs with different random seeds.Source: §5.1 · pp. 5–6, Table 2 · p. 6

5.2 Main Results

• REWIND achieves the highest mean accuracy in all 15 model–benchmark settings; on Qwen2.5-Math-7B/MATH500, it scores 77.60% versus 72.60% for standard MCMC.The comparison adds one entropy-guided revision after the shared K = 10 random MCMC steps per block.Source: Table 2 · p. 6, §5.2 · p. 6

5.3 Signal Comparison and Analysis

• On Qwen2.5-Math-7B/MATH500, entropy-gradient backtracking reaches 77.60%, versus 74.20% for peak revision and 72.60% for random-proposal MCMC.At the same proposal-count budget, it exceeds Random-in-Window by 2.40 points; the ablation uses K = 10 random MCMC steps per block.Source: §5.3 · pp. 6–7, Table 3 · p. 7
• Boundary alignment raises accuracy from 75.00% to 77.60%, while restricting delimiters to periods and newlines retains 76.80%.Random Delimiter revision reaches 73.20%, indicating that boundary cues work best with targeted site selection.Source: §5.3 · p. 7
• Tracing improves MATH500 accuracy for all six tested signals, with gains from 0.2 points for skewness to 3.4 for entropy.Across three benchmarks, entropy performs best among tested triggers; alternatives show uneven transfer, including lower AIME2025 accuracy than MCMC for Top-K similarity and kurtosis.Source: §5.3 · p. 7, Table 4 · p. 7, Table 5 · p. 7

5.4 Inference Budget and BoN Comparison

• REWIND exceeds likelihood-ranked BoN@10 on all 15 reported model–benchmark pairs, with gains of 1.60–11.47 percentage points.REWIND adds approximately 4.8% generated tokens over MCMC.Source: §5.4 · p. 8

5.5 Transfer across Tasks and Prompts

• On HumanEval, REWIND gains 2.4 points over MCMC on Qwen2.5-Math-7B and 3.7 on Tülu-8B; with 8-shot CoT, it gains 5.6, 2.1, and 6.7 points on MATH500, GSM8K, and AIME2025.The two-delimiter Qwen2.5-Math-7B HumanEval variant reaches 57.90%, compared with 58.5% for the full delimiter set and 56.1% for MCMC.Source: §5.5 · p. 8, Table 6 · p. 8

• Mean token entropy best distinguishes response correctness, while token-wise entropy trajectories retain the positional information used to select revision sites.REWIND anchors a local search at an entropy peak and selects the largest entropy increase within that window.Source: §6 · p. 8
• Entropy is the strongest cross-benchmark signal, tracing improves over peak revision for all six tested signals on MATH500, and boundary alignment remains effective with periods and newlines.The discussion identifies both an informative uncertainty signal and a rule for converting it into a suffix proposal as ingredients of effective revision.Source: §6 · p. 8
• Manual inspection finds confident errors without a salient entropy spike in 7.6% of problems and detected but unrecovered failures in 14.8%.The unrecovered cases account for most residual failures, leaving regeneration quality as a bottleneck.Source: §6 · p. 8

• Incorrect responses have higher mean token entropy on 88.12% of paired MATH500 questions, with an average within-question relative increase of 112%.Source: §7 · p. 8
• REWIND uses entropy-gradient backtracking, natural-boundary alignment, and an MH-style score to select and filter regenerated suffixes without additional training.Source: §7 · p. 8
• Across three model families and five benchmarks, REWIND outperforms all baselines, improving over random-proposal MCMC by 0.51–10.00 percentage points.Signal comparisons favor entropy among tested triggers, while controlled ablations support gradient backtracking and boundary alignment.Source: §7 · p. 8

Computational Cost

• The cost study counts generation tokens but excludes entropy-only forward passes and does not establish normalized FLOPs or wall-clock cost.Entropy computation and discarded proposals add overhead; efficiency comparisons depend on implementation and batching.Source: Limitations · p. 9

Scope and Generalization

• Task and model coverage is limited, and supplementary HumanEval results cover only two backbones.Further tasks, backbones, and repeated runs are needed to assess broader transfer.Source: Limitations · p. 9

Signal Validity and Statistical Coverage

• Benchmarks lack token-level first-error annotations, AIME2025 has 30 questions, and 7.6% of inspected MATH500 examples are confidently wrong without a salient entropy spike.The evidence therefore concerns intervention utility rather than localization accuracy, and the AIME2025 accuracy comparisons have limited precision.Source: Limitations · p. 9

Ethical Considerations

• REWIND can still produce incorrect answers, and iterative revision requires additional inference computation that may increase energy consumption.Source: Ethical Considerations · p. 9

Appendix Overview

• Appendices provide theoretical analysis, robustness checks, failure analyses, supplementary evaluations, case studies, and implementation details for the main claims.Source: Appendix Overview · p. 11

A.2 Main Result

• The entropy-gradient analysis models uncertainty as within-interpretation entropy plus cross-interpretation disagreement, with an isolated disagreement jump producing a detectable gradient spike under stated conditions.The theorem assumes locally smooth within-interpretation entropy and no comparable mutual-information jumps at competing positions in the search window.Source: §A.2 · p. 11

A.3 Why an Earlier Cut May Help

• Backtracking before a later entropy peak removes intervening tokens and can provide an opportunity to revise a harmful premise; its benefit depends on the removed content.Source: §A.3 · pp. 11–12

A.4 When Does Fragmentation Produce a Detectable Signal?

• The appendix’s two-world lemma links prediction disagreement between plausible interpretations to a larger mutual-information component, whose isolated jump can support temporal localization.The lower bound depends on the worlds’ probability mass, minimum conditional weights, and prediction disagreement.Source: §A.4 · p. 12

A.5 Scope of the Analysis

• The analysis covers errors that manifest as localized semantic fragmentation; confident coherent errors and gradual uncertainty drift can weaken the localized-jump signal.Large uncertainty shifts during legitimate reasoning transitions can also compete with the signal and affect the selected revision site.Source: §A.5 · p. 12

A.6 Sequence-Level Score Motivation

• REWIND compares suffix scores at the selected site with an MH-style filter because prefix-dependent temperature normalizers vary across trajectories, while exact MH cancels the global normalizer.The state-dependent site-selection probabilities are omitted from the filter.Source: §A.6 · p. 12

B Comparison with Qwen2.5-Math-72B

• With a Qwen2.5-Math-7B backbone, REWIND outperforms single-pass Qwen2.5-Math-72B on MATH500, GPQA, AIME2025, and BBH, while the 72B model scores higher on GSM8K.The comparison trades model scale against iterative inference computation.Source: B Comparison with Qwen2.5-Math-72B · p. 12

D Signal Robustness

• Peak selection maximizes a metric value, whereas gradient tracing selects the largest metric increase within the preceding peak window.The evaluated statistics include KL divergence, Top-K similarity, skewness, and kurtosis; signal variants share the suffix-generation and acceptance procedure.Source: D Signal Robustness · pp. 13–14
• Entropy is the top trigger on all three benchmarks, while gradient tracing improves every listed configuration on MATH500, with gains of 3.4 points for entropy and 3.2 for temporal KL.Source: D Signal Robustness · p. 14

E Supplementary Controller Robustness

• Maximum probability matches entropy on AIME2025 but falls below MCMC on MATH500, while probability gap and varentropy also trail entropy on MATH500.Source: E Supplementary Controller Robustness · p. 14
• The default delimiter set and fixed w = 5 window achieve the highest accuracy; expanded delimiters and an adaptive window reduce accuracy relative to that setting.Source: E Supplementary Controller Robustness · p. 14

F Answer-Level Intervention Analysis

• REWIND repairs 46 previously incorrect MATH500 answers and regresses on 21 previously correct answers per run on average, for a net gain of 25 correct answers per 500 questions.This corresponds to the 5.00-percentage-point difference in mean accuracy reported in the main table.Source: F Answer-Level Intervention Analysis · p. 14, Table 12 · p. 15
• On MATH500, the complete controller accepts 54.1% of proposals, versus 57.3% for random MCMC, alongside its positive net answer gain.Source: F Answer-Level Intervention Analysis · p. 14

G Intervention-Site Coverage

• On MATH500, the controller visits an average of 3.18 distinct sequence positions per problem across refinement rounds.The statistic summarizes distinct aligned cut points visited under the blockwise protocol.Source: G Intervention-Site Coverage · p. 14, Table 14 · p. 15

H Residual Failure Analysis

• Detected-but-unrecovered cases account for 66.1% of residual failures, motivating stronger suffix proposals for detected instability and complementary signals for confident errors.Source: H Residual Failure Analysis · p. 14

I Standard Few-Shot CoT Evaluation

• Under standard 8-shot CoT prompting, REWIND continues to improve over vanilla decoding, temperature sampling, and standard MCMC on the math benchmarks.Source: I Standard Few-Shot CoT Evaluation · pp. 14–15

J Additional Generalization Results

• Supplementary HumanEval results show gains beyond mathematics for Qwen2.5-Math-7B and Tülu-8B.Source: J Additional Generalization Results · p. 15

K Multi-Seed Stability

• The multi-seed appendix reports arithmetic means and sample standard deviations of run-level accuracies for Qwen2.5-Math-7B on MATH500 and AIME2025 across the same three runs as the main results.Source: K Multi-Seed Stability · p. 15

L Case Studies

• The case studies illustrate how REWIND identifies and corrects problematic reasoning patterns.Source: L Case Studies · p. 15

L.1 Case Study 1: Methodological Improvement

• The revision replaces exhaustive checking with a number-theoretic solution that reduces the problem to finding divisors of 56.The example begins with the question of which integer bases b > 9 satisfy the stated divisibility condition.Source: §L.1 · pp. 15–16

L.2 Case Study 2: Geometric Error Correction

• The geometric case corrects the setup by placing F on AC rather than the x-axis representing AB.The case summary characterizes this as fixing a fundamental setup error before it propagates.Source: §L.2 · pp. 16–17

M.1 Benchmark Subsets and Prompts

• The benchmark appendix defines evaluation subsets, including all 30 AIME2025 questions, BBH’s formal_fallacies task, and GPQA’s Diamond subset.Accuracy is computed over the questions in each run’s listed subset.Source: §M.1 · p. 17

M.2 Best-of-N Sampling and Ranking

• BoN@10 independently samples ten responses per input and ranks candidates using original conditional probabilities, excluding prompt, padding, and EOS tokens.Empty responses are excluded; candidates are not filtered by answer validity or correctness, and only the selected response contributes to single-output accuracy.Source: §M.2 · pp. 17–18

M.3 Controller Implementation

• Each controller step selects an entropy peak, aligns the regeneration boundary to a nearby delimiter when available, and proposes a suffix revision.Repeated steps recompute the retained sequence’s entropy trajectory; the nearest matching delimiter is sought within the preceding 10 generated tokens.Source: §M.3 · p. 18