PaperlayerView exact revision on arXiv

Source-linked AI summary

Not All Queries Need Deep Thought: CoFiCot for Adaptive Coarse-to-fine Stateful Refinement

Dongxu Zhang, Hongqiang Lin, Yiding Sun, Pengyu Wang, Qirui Wang, Ning Yang, Jihua Zhu

arXiv:2603.08251v1cs.CL

TL;DR

Uniformly allocating computation can overthink simple queries and under-refine complex ones. CoFiCot triages queries by difficulty and applies differentiated refinement; across seven benchmarks, it outperforms strong baselines and achieves a better accuracy-efficiency trade-off.

  • Problem

    Uniformly allocating computation across queries is misaligned with the variable difficulty of reasoning tasks.

    Source: §1 · p. 1

  • Method

    CoFiCot triages queries by difficulty, aggregates easy cases, and applies stateful sequential correction to medium and hard cases.

    Source: §1 · p. 2

  • Results

    Across seven benchmarks, CoFiCot outperforms strong baselines; for 3-8B-Instruct, average accuracy is 75.0%, a 4.0% absolute improvement over Best-of-k.

    Source: §1 · p. 2, §4.2 · p. 6

  • Takeaways & Limitations

    The paper concludes that dynamic early exit preserves computation once reasoning stabilizes, avoiding redundant iterations.

    Source: §5 · p. 8

    Effectiveness depends on external models guiding refinement and selection, while generating k samples adds overhead for extremely simple queries.

    Source: §6 · p. 9

Abstract

from arXiv · show

Scaling test-time computation enhances LLM reasoning ability but faces a uniform computation paradox. Allocating identical resources leads to over-correction on simple tasks and insufficient refinement on complex ones. To address this, we propose CoFiCot, a coarse-to-fine adaptive framework that dynamically tailors inference strategies to problem difficulty. Specifically, we implement a multi-metric classifier that triages queries by synthesizing semantic entropy, consensus reliability, and predicted reasoning depth . This enables a differentiated refinement stage that applies efficient aggregation for simple queries while routing complex ones to a context-aware correction loop . We formalize correction as a stateful sequential propagation process , where each repair is strictly conditioned on the verified history of prior rectifications. By integrating Process Reward Models (PRMs) within this state-dependent trajectory, CoFiCot effectively bridges the gap between granular error localization and global logical coherence, preventing the context fragmentation typical of stateless refinement methods.

1 INTRODUCTION

CoFiCot addresses the mismatch between uniform test-time computation and queries’ varying difficulty with adaptive routing and stateful sequential correction. It uses efficient aggregation for easy queries and iterative refinement for medium and hard queries.

1 INTRODUCTION

  • Uniform computation can overthink simple queries and under-refine complex ones, with unnecessary iterations corrupting correct answers and fixed budgets ending complex reasoning prematurely.The introduction also notes saturation and hallucination on simple tasks and brittleness from single errors in complex reasoning chains.Source: §1 · pp. 1–2, Figure 1 · p. 1
  • Routing strategies can improve efficiency but lack active error repair, while stateless iterative refinement can break downstream logic when an intermediate step changes [Xia et al., 2025, Madaan et al., 2023].The introduction frames these as limitations of existing adaptive approaches.Source: §1 · p. 2
  • CoFiCot triages queries into Easy, Medium, or Hard, using efficient aggregation for Easy queries and iterative correction for Medium and Hard ones.The framework is designed to dynamically match reasoning strategies to problem difficulty.Source: §1 · p. 2
  • Stateful Sequential Correction anchors the validated reasoning path and branches from an error so corrections update dependent downstream steps.This history-generative process aims to combine step-level error localization with coherence across complex reasoning.Source: §1 · p. 2
  • CoFiCot is reported to outperform strong baselines across seven benchmarks and achieve a better trade-off between accuracy and computation.The supplied passage does not provide numerical results or complete the trade-off description.Source: §1 · p. 2

2 RELATED WORK

Prior reasoning, efficiency, and feedback methods face limits including performance saturation, mismatched computation for different query complexities, and loss of global coherence in repair. CoFiCot addresses these challenges with adaptive difficulty assessment and stateful refinement.

2.1 ENHANCING REASONING ABILITY IN LLMS

2.2 COMPUTATIONAL EFFICIENCY IN LLMS

  • Static compression methods reduce per-token costs but fail to account for query complexity, while C2F methods trade latency against corrective capability.MAgICoRe [Chen et al., 2024a] incurs prohibitive latency, whereas early exit strategies [Zhang et al., 2025] do not actively repair flawed reasoning; CoFiCot combines granular difficulty assessment with stateful refinement.Source: §2.2 · p. 2

2.3 FEEDBACK-DRIVEN ERROR RECTIFICATION

  • Intrinsic self-correction struggles to diagnose errors, and PRM-guided repairs can lack global coherence; CoFiCot formalizes correction as propagation to maintain reasoning integrity.PRMs provide step-level signals, while ORMs evaluate entire traces; CoFiCot conditions correction on verified prior steps and uses an ORM to select refined solutions.Source: §2.3 · pp. 2–3, Figure 2 · p. 3

3 METHODOLOGY

CoFiCot samples an ensemble of reasoning traces, classifies query difficulty using consensus confidence, consensus reliability, and predicted reasoning complexity, then routes queries to aggregation or iterative refinement. For harder queries, PRM-guided correction preserves verified steps and regenerates subsequent reasoning from the corrected state.

3.1 OVERVIEW AND DATA PREPARATION

  • CoFiCot begins with k independent chain-of-thought candidates and routes each query, after difficulty assessment, either to immediate aggregation or iterative refinement.The initial candidate set is R0 = {r1, . . . , rk}; the workflow is illustrated in Figure 2, with execution details in Algorithm 1 in Appendix A.Source: §3.1 · p. 3

3.2 STAGE 1: COARSE-GRAINED CLASSIFICATION

  • The classifier combines semantic-consensus confidence, majority-cluster reliability, and predicted reasoning-step complexity into a weighted final difficulty score.Confidence is derived from answer-cluster entropy; reliability compares the majority cluster’s mean reward-model score with the candidate pool’s global distribution, while complexity is estimated by prompting the base LLM for required reasoning steps.Source: §3.2 · pp. 3–4
  • The reliability score labels consensus Easy when its normalized reward score reaches δ, Medium when it is nonnegative but below δ, and Hard when negative.This thresholding treats consensus quality as a check against false consensus, complementing the entropy-based confidence filter.Source: §3.2 · p. 4

3.3 STAGE 2: FINE-GRAINED REFINEMENT

  • Medium and Hard queries enter a PRM-guided loop that flags low-scoring steps, preserves verified history, corrects the first error, and regenerates later steps.Corrections are conditioned on the question and verified history; Easy queries bypass the refinement loop and use the initial ensemble.Source: §3.3 · p. 4
  • After each correction round, an ORM scores the expanded candidate pool, the top k candidates are retained, and refinement stops if difficulty becomes Easy or the iteration budget is exhausted.The retained set is re-evaluated by the Stage 1 classifier at each iteration.Source: §3.3 · p. 5

4 EXPERIMENTS

Across seven benchmarks, CoFiCot is evaluated for accuracy and sampling or token efficiency, with additional analyses examining model and reward-model choices, pipeline components, and a traced correction case. Results show gains over strong baselines, while ablations and the case study probe how its stages and components contribute.

4.1 EXPERIMENTAL SETUP

  • The evaluation spans seven datasets, two backbone models, and compares CoFiCot with prompting and aggregation baselines using accuracy and average generated samples per question (k).The baselines include zero-shot CoT, k-way Self-Consistency, and ORM-based Best-of-k; aggregation baselines use k = 120 samples.Source: §4.1 · p. 5

4.2 MAIN RESULTS

  • At k = 40, CoFiCot achieves 45.9% accuracy on MATH, exceeding k-way SC (40.6%) and Best-of-k (41.4%) at k = 120.Both aggregation baselines plateau as k increases from 40 to 120, while CoFiCot scales more effectively with increased computational budget.Source: Figure 3 · p. 5, §4.2 · p. 6
  • CoFiCot averages 75.0% on Llama-3-8B-Instruct (4.0% above Best-of-k) and 80.5% on GPT-3.5-Turbo (3.2% above).On ARC it reaches 88.2%, 2.2% above SC 120-ways; on Date Understanding, it reaches 80.8% versus 72.5% for 120-way SC.Source: Table 1 · p. 6, §4.2 · p. 6, Table 2 · p. 6

4.3 ADDITIONAL ANALYSIS

  • On Qwen2.5-Math-7B, CoFiCot averages 93.4% accuracy; on MATH, upgrading the PRM raises accuracy from 48.3% to 54.9% (+6.6%).Upgrading the ORM from InternLM-7B to Llama-3.1-8B raises accuracy from 48.3% to 51.2%, demonstrating sensitivity to reward-model quality.Source: §4.3 · pp. 6–7, Table 3 · p. 7, Table 4 · p. 7

4.5 ABLATION STUDIES

  • Without the Coarse Stage, GSM8K accuracy drops 3.3%; without the Fine Stage, MATH accuracy drops 6.7%.The ablation associates the Coarse Stage with avoiding harmful over-correction on simpler problems and the Fine Stage with solving difficult problems.Source: Table 5 · p. 7, §4.5 · p. 8

4.4 CASE STUDY

  • In the APPLE permutation case, low-quality initial solutions lead the classifier to label the problem Hard and route it to fine-grained refinement.The PRM localizes an error in Step 3; correction uses targeted feedback and validated prior steps to generate a corrected solution, which the ORM scores for the next top-k set.Source: §4.4 · pp. 7–8

4.3 ADDITIONAL ANALYSIS

  • CoFiCot achieves 47.9% accuracy on MATH using fewer tokens than 120-way SC and uses a fraction of its tokens on MMLU and SAT while achieving higher accuracy.Although sequential refinement may increase latency per token, bypassing it for approximately 40–60% of Easy queries keeps average latency competitive.Source: §4.3 · p. 7, Figure 4 · p. 8

4.5 ABLATION STUDIES

  • Adding the Cofi. metric to Relia. and Comp. raises GSM8K accuracy from 89.7% to 91.8% and MMLU from 67.0% to 68.8%.All three classification metrics contribute positively; variants using one or two underperform the full model across datasets.Source: Table 6 · p. 8, §4.5 · p. 8

5 CONCLUSION

CoFiCot adapts reasoning refinement to query difficulty and uses stateful PRM-guided correction with early exit. Across seven benchmarks, it outperforms uniform scaling baselines while improving accuracy and token efficiency.

5 CONCLUSION

  • 4.0% average accuracy gain on Llama-3-8B across seven benchmarks, with superior token efficiency over uniform scaling baselines.The results establish a stronger trade-off between accuracy and computational cost.Source: §5 · p. 8
  • CoFiCot triages problems using multi-metric assessment and applies differentiated refinement to mitigate over-correction on simple tasks and insufficient refinement on complex ones.Source: §5 · p. 8
  • Its sequential correction mechanism uses PRMs to initiate decoding branches from localized error points, supporting logically coherent repairs.The framework can integrate different reward models, and performance scales with their quality.Source: §5 · p. 8
  • A dynamic early exit protocol preserves computation once reasoning stabilizes, avoiding redundant iterations.Source: §5 · p. 8
  • Future work will automate parameter calibration and extend CoFiCot to broader domains.Source: §5 · p. 8

6 LIMITATION

CoFiCot depends on external models to guide refinement and selection, and sampling k responses adds overhead for extremely simple queries compared with greedy decoding.

6 LIMITATION

  • CoFiCot depends on external models for refinement and selection, while generating k samples adds overhead on extremely simple queries compared with greedy decoding.The authors suggest progressive sampling as future work.Source: §6 · p. 9
  • Progressive sampling is proposed as a future strategy to address the overhead of generating k samples for extremely simple queries.Source: §6 · p. 9

A ALGORITHM

CoFiCot generates a diverse solution ensemble, classifies query difficulty from multiple signals, and routes solutions through difficulty-dependent refinement and aggregation. Its iterative path uses PRM- and ORM-guided selection and exits early when a query becomes Easy.

A ALGORITHM

  • CoFiCot samples an initial solution ensemble, then classifies difficulty using entropy, consensus quality, and predicted reasoning steps.The algorithm names these stages GenerateSolutions and ClassifyDifficulty, respectively.Source: A ALGORITHM · p. 12
  • Easy problems bypass refinement and proceed directly to final aggregation, while Medium and Hard problems enter an iterative refinement loop.This routing is intended to minimize computational overhead for Easy problems.Source: A ALGORITHM · p. 12
  • ContextAwareCorrection uses a PRM to localize and regenerate error steps based on prior corrections, while SelectTopK uses an ORM to retain the highest-quality candidates.SelectTopK filters the expanded candidate pool for the next iteration.Source: A ALGORITHM · p. 12
  • The refinement loop exits early when reevaluated difficulty becomes Easy, avoiding redundant iterations before final WeightedVoting aggregation.Source: A ALGORITHM · p. 12

B DETAILED IMPLEMENTATION OF COFICOT

CoFiCot assesses difficulty before refinement, skips refinement for Easy cases, and applies context-aware correction to other cases. Its implementation combines weighted self-consistency with a stateful correction process.

B DETAILED IMPLEMENTATION OF COFICOT

  • Entropy H(A) is converted to normalized confidence C ∈[0, 1] by a scaled sigmoid with α = 2, while a metacognitive look-ahead prompt estimates Nsteps without full chain generation.The complexity-prediction prompt is presented in Table 7.Source: B DETAILED IMPLEMENTATION OF COFICOT · p. 12, Table 7 · p. 14
  • Algorithm 1 generates k diverse solutions, classifies difficulty, skips refinement for Easy cases, and iteratively corrects, selects top-k, and reclassifies other cases before weighted self-consistency voting.The loop stops when difficulty is reclassified as Easy or the maximum iterations are reached.Source: B DETAILED IMPLEMENTATION OF COFICOT · p. 13
  • Less than ±0.5% accuracy variance on MATH results from changing the uniform classification weights; α ∈[1.5, 2.5] yields statistically similar classification boundaries.The reported weight sensitivity supports the robustness of the metric ensemble, while the entropy-scaling range reflects similar classification boundaries.Source: B DETAILED IMPLEMENTATION OF COFICOT · p. 13
  • Stage 1 maps Easy, Medium, and Hard to ranks 1, 2, and 3, then uses their weighted average, with uniform weights w_i = 1/3, to select the routing path.Easy and Hard thresholds derive from the 33rd and 67th percentiles of base-model solution lengths, adapting to different models or prompt styles without manual recalibration.Source: B DETAILED IMPLEMENTATION OF COFICOT · pp. 12–13
  • Φ treats verified history before step j as a fixed prefix, converts PRM feedback into instruction, and uses each rectified successor as the next autoregressive decoding state.The model then generates the remaining trajectory from that new state.Source: B DETAILED IMPLEMENTATION OF COFICOT · pp. 13–14

C ANALYSIS

The analysis distinguishes total compute from user-perceived latency: CoFiCot parallelizes its initial sample ensemble, while sequential refinement can increase latency. It also notes the energy cost of sampling more than once and describes progressive sampling as a resource-saving option.

Computational Overhead and Latency Analysis

  • O(1) latency for generating k = 40 initial samples is comparable to one sequence when GPU memory is sufficient, and easy queries bypass Stage 2.The passage contrasts this parallel generation with sequential heavy reasoning or iterative refinement, which it characterizes as O(N).Source: C ANALYSIS · p. 14
  • k = 40 sampling uses more energy than k = 1 greedy decoding, but ensemble consistency checks filter confident errors that greedy decoding risks producing.The passage says the confidence assessment ablation finds the consistency check crucial for filtering such errors.Source: C ANALYSIS · p. 14
  • Resource-constrained deployments can start with k = 5 samples and scale to k = 40 only when confidence falls below a safety threshold.Source: C ANALYSIS · p. 14

D ABLATION EXPERIMENTS

CoFiCot’s balanced difficulty aggregation achieved the best reported results on GSM8K and MATH, while the fine-stage ablation supports using PRM feedback for correction and ORM scores for selection.

Analysis of Difficulty Aggregation Strategy

  • CoFiCot’s balanced averaging achieved the best performance: 91.8% on GSM8K and 47.9% on MATH, ahead of Pessimistic, Democratic, and Optimistic aggregation.Pessimistic was a close second; Democratic and Optimistic performed worst because they were more likely to classify hard problems as easy and skip refinement.Source: D ABLATION EXPERIMENTS · p. 15

Synergy of PRM and ORM in Refinement

  • The full dual-RM model achieved the highest accuracy, using PRM step-level feedback for correction and ORM solution-level scores for selection.ORM-Only relied on LLM self-feedback for correction, while PRM-Only used aggregated step scores for selection; both variants performed worse.Source: D ABLATION EXPERIMENTS · p. 15

E REFINEMENT IMPLEMENTATION DETAILS · F MODULARITY AND GENERALIZATION · G EXTENDING ADAPTIVE REASONING TO MULTIMODAL HEALTHCARE

CoFiCot uses weighted aggregation for Easy problems and dynamically stops iterative refinement when solutions become Easy or reach difficulty-specific iteration limits. Its modular verifier and reward-model configuration supports sequential correction across reasoning domains, while multimodal healthcare applications are proposed as future work.

E REFINEMENT IMPLEMENTATION DETAILS

  • For Easy problems, CoFiCot selects the final answer using Weighted Self-Consistency, weighting each vote by the solution’s ORM score.This avoids the iterative loop while supporting robust aggregation.Source: E REFINEMENT IMPLEMENTATION DETAILS · p. 15
  • The Stage 2 loop stops when the Stage 1 classifier reclassifies the solution set as Easy or when it reaches 2 iterations for Medium or 3 for Hard problems.Source: E REFINEMENT IMPLEMENTATION DETAILS · p. 15

F MODULARITY AND GENERALIZATION

  • CoFiCot’s framework does not require a domain-specific PRM, although its MATH and GSM8K experiments use Math-Shepherd-7B.Source: F MODULARITY AND GENERALIZATION · p. 15
  • For Commonsense Reasoning with scarce dense step-level supervision, CoFiCot uses a prompted general-purpose LLM as verifier and relies more heavily on the ORM for Stage 2 trajectory selection.The ablation study reports that the ORM alone contributes significantly to performance.Source: F MODULARITY AND GENERALIZATION · p. 15
  • With general-purpose verifiers, CoFiCot is reported to generalize to Date Understanding and ARC, preserving logical coherence during sequential correction.The passage points to Table 2 for the generalization results.Source: F MODULARITY AND GENERALIZATION · p. 15

G EXTENDING ADAPTIVE REASONING TO MULTIMODAL HEALTHCARE

  • Future work could extend stateful adaptive refinement to multimodal clinical evaluations, including cognitive tests with wearable streams and imaging fused with genomic profiles.The passage frames these as potential applications for early disease screening and cancer mutation-status prediction.Source: G EXTENDING ADAPTIVE REASONING TO… · p. 16

Related papers

Loading 2603.08251v1…