PaperlayerView exact revision on arXiv

Source-linked AI summary

Not All Queries Need Deep Thought: CoFiCot for Adaptive Coarse-to-fine Stateful Refinement

Dongxu Zhang, Hongqiang Lin, Yiding Sun, Pengyu Wang, Qirui Wang, Ning Yang, Jihua Zhu

arXiv:2603.08251v1cs.CL

TL;DR

Uniform test-time computation can over-correct simple queries while under-refining complex ones. CoFiCot uses multi-metric difficulty triage, differentiated inference, and stateful PRM-guided correction; it reports stronger benchmark performance and a better accuracy-efficiency trade-off, with latency and sampling overhead as boundaries.

  • Problem

    Uniform resource allocation misaligns inference effort with query difficulty, causing over-correction on simple tasks and insufficient refinement on complex reasoning.

  • Method

    CoFiCot synthesizes semantic confidence, consensus reliability, and predicted reasoning depth to route easy queries to aggregation and harder queries to context-aware sequential correction.

  • Results

    CoFiCot outperforms strong baselines across seven benchmarks and achieves a better accuracy-efficiency trade-off; on Llama-3-8B-Instruct, average accuracy is 75.0%, 4.0% above Best-of-k.

  • Takeaways & Limitations

    Stateful correction preserves validated reasoning history while propagating updates downstream, combining localized error repair with global logical coherence.

  • Takeaways & Limitations

    The framework depends on external models for refinement and selection, and generating k samples can add overhead for extremely simple queries.

Abstract

from arXiv · show

Scaling test-time computation enhances LLM reasoning ability but faces a uniform computation paradox. Allocating identical resources leads to over-correction on simple tasks and insufficient refinement on complex ones. To address this, we propose CoFiCot, a coarse-to-fine adaptive framework that dynamically tailors inference strategies to problem difficulty. Specifically, we implement a multi-metric classifier that triages queries by synthesizing semantic entropy, consensus reliability, and predicted reasoning depth . This enables a differentiated refinement stage that applies efficient aggregation for simple queries while routing complex ones to a context-aware correction loop . We formalize correction as a stateful sequential propagation process , where each repair is strictly conditioned on the verified history of prior rectifications. By integrating Process Reward Models (PRMs) within this state-dependent trajectory, CoFiCot effectively bridges the gap between granular error localization and global logical coherence, preventing the context fragmentation typical of stateless refinement methods.

1 INTRODUCTION

Test-time scaling creates a uniform computation paradox: extra reasoning can corrupt easy answers, while fixed budgets can truncate difficult reasoning. CoFiCot addresses this by routing queries to difficulty-appropriate aggregation or stateful correction.

  • Motivation: Test-time scaling improves reasoning, but uniform computation misaligns resources with variable task complexity.Additional compute can yield gains comparable to parameter scaling, motivating adaptive allocation.
  • Motivation: Easy tasks may suffer overthinking and hallucination, while complex tasks remain insufficiently refined when reasoning chains contain errors.The paper characterizes these outcomes as over-correction and insufficient refinement.
  • Gap: Existing routing improves efficiency but does not actively repair errors, while stateless refinement can invalidate later reasoning after an intermediate correction.These limitations motivate combining adaptive allocation with context-aware repair.
  • CoFiCot: CoFiCot triages queries into Easy, Medium, or Hard and applies aggregation to easy problems while routing harder problems into iterative correction.Its coarse-to-fine pipeline dynamically matches inference strategy to problem difficulty.
  • CoFiCot: Stateful sequential correction preserves validated history and propagates repairs through dependent reasoning steps to maintain logical consistency.The mechanism starts a new decoding branch from the error point rather than regenerating the entire chain.
  • Results: Experiments across seven benchmarks report that CoFiCot outperforms strong baselines while achieving a better accuracy-efficiency trade-off.The supplied introduction states the cross-benchmark result without reporting a numerical value.

2 RELATED WORK

Prior work explores ensemble aggregation, iterative refinement, adaptive computation, and reward-model feedback, but these approaches retain complementary limitations that CoFiCot combines into one workflow.

  • Ensemble inference: Self-Consistency and Best-of-k reduce single-path fragility through aggregation, yet their costs rise linearly while accuracy gains plateau.These methods select consensus or high-reward answers from multiple generated candidates.
  • Iterative refinement: Indiscriminate iterative refinement can corrupt correct simple-task answers and inadequately repair complex reasoning failures.The related-work discussion concludes that iteration alone is insufficient.
  • Adaptive computation: Compression and coarse-to-fine methods reduce computation or tailor allocation, but static paths and latency-correction trade-offs remain concerns.The passage frames adaptive computation as promising while noting unresolved constraints.
  • Reward feedback: Intrinsic self-correction often fails to diagnose errors reliably, motivating external feedback from specialized reward models.The related work describes a shift toward externally guided refinement.
  • CoFiCot: CoFiCot’s workflow combines multi-metric routing, simple aggregation for easy cases, PRM-guided correction, and ORM-based selection for harder cases.The figure describes iterative repetition until termination criteria are met.
  • Reward feedback: Outcome reward models score complete traces, whereas process reward models provide dense step-level signals but can lack global coherence when integrated into refinement.CoFiCot addresses this by formalizing correction as a propagation process.

3 METHODOLOGY

CoFiCot first classifies query difficulty from complementary signals, then either aggregates the initial candidate ensemble or performs stateful, PRM-guided sequential refinement with adaptive termination.

  • Stage 0: The pipeline generates an ensemble of k candidate solutions before coarse-grained difficulty analysis routes queries to aggregation or refinement.The initial candidate set R0 provides the basis for subsequent classification and inference.
  • Stage 1: Coarse-Grained Classification: Stage 1 classifies problems as Easy, Medium, or Hard by synthesizing three complementary metrics.The metrics assess semantic confidence, consensus reliability, and predicted reasoning complexity.
  • Stage 1: Coarse-Grained Classification: Semantic confidence is estimated from the entropy of semantic answer clusters and transformed into a normalized confidence score.The score is used to assign a preliminary difficulty label through empirical thresholds.
  • Stage 1: Coarse-Grained Classification: Consensus reliability compares the majority cluster’s reward quality with the global candidate distribution using a normalized score.A negative normalized score indicates that the popular consensus underperforms the candidate pool average.
  • Stage 1: Coarse-Grained Classification: Predicted reasoning depth estimates required steps with a metacognitive prompt, using corpus-based thresholds to assign a difficulty label at negligible overhead.The prediction requires generating only a single integer token.
  • Stage 1: Coarse-Grained Classification: The final difficulty score is a weighted sum of mapped metric labels, with each weight representing a metric’s relative importance.The mapping converts qualitative difficulty labels into discrete numerical ranks.
  • Stage 2: Fine-Grained Refinement: Easy problems bypass the expensive refinement loop and are resolved directly from the initial ensemble.Medium and Hard problems proceed to the fine-grained refinement stage.
  • Stage 2: Fine-Grained Refinement: For Medium and Hard problems, a PRM identifies the first low-scoring step, after which correction conditions on the question and verified preceding history.This state-dependent transformation uses the frozen history before the detected error.

4 EXPERIMENTS

Across mathematical and general reasoning benchmarks, CoFiCot consistently improves accuracy while reducing or controlling computational cost through adaptive refinement. Ablations further support the value of its coarse-to-fine stages, classification metrics, and modular reward models.

  • Experimental setup: CoFiCot is evaluated across seven benchmarks, two backbone models, and mathematical, commonsense, and logical reasoning settings.The experiments compare against strong aggregation and prompting baselines using accuracy and adaptive sample or token costs.
  • Main results: 75.0% average accuracy on Llama-3-8B-Instruct is 4.0% above Best-of-k, while 80.5% on GPT-3.5-Turbo exceeds its strongest baseline by 3.2%.On Llama-3-8B-Instruct, CoFiCot also gains 6.5% over the 41.4% Best-of-k result on MATH.
  • Main results: 45.9% accuracy at k = 40 lets CoFiCot exceed k-way Self-Consistency at 40.6% and Best-of-k at 41.4%, each using k = 120.The comparison illustrates stronger scaling with increased computational budget than the aggregation baselines on MATH.
  • Generalization: CoFiCot reaches 88.2% on ARC and 80.8% on Date Understanding, exceeding 120-way Self-Consistency by 2.2% and 8.3%, respectively.These results extend the reported gains beyond mathematical reasoning tasks.
  • Additional analysis: CoFiCot achieves 93.4% average accuracy on Qwen2.5-Math-7B, with 91.7% on MATH, indicating gains beyond the weaker backbone setting.The analysis presents the framework as scalable to a stronger mathematical reasoning model.
  • Additional analysis: Replacing Math-Shepherd-7B with Qwen-Math 7B raises accuracy from 48.3% to 54.9%, while replacing InternLM-7B with Llama-3.1-8B raises it to 51.2%.The component-swap analysis supports CoFiCot’s ability to benefit from stronger reward models.
  • Efficiency: CoFiCot achieves higher accuracy with fewer or comparable tokens than 120-way Self-Consistency, while bypassing refinement for approximately 40–60% of Easy queries keeps average latency competitive.The sequential refinement loop can increase latency per token, but adaptive routing reduces average computational load and deployment cost.
  • Ablation studies: Removing the Coarse Stage reduces GSM8K accuracy by 3.3%, whereas removing the Fine Stage reduces difficult-task MATH accuracy by 6.7%.The ablation attributes the coarse stage to avoiding indiscriminate refinement and the fine stage to handling complex problems.

5 CONCLUSION

CoFiCot addresses inefficient uniform test-time computation through adaptive triage, differentiated refinement, and stateful PRM-guided correction. Across seven benchmarks, it reports higher accuracy and token efficiency than uniform scaling baselines.

  • CoFiCot triages problems with multi-metric assessment and applies differentiated refinement to mitigate over-correction on simple tasks and insufficient refinement on complex ones.Its sequential correction mechanism uses PRMs to maintain logically coherent repairs from localized error points, while dynamic early exit avoids redundant iterations.
  • 4.0% average accuracy gain on Llama-3-8B is reported across seven benchmarks while maintaining superior token efficiency against uniform scaling baselines.
  • CoFiCot supports modular integration of reward models, with performance scaling directly alongside the quality of these components.

6 LIMITATION

The pipeline depends on external models for refinement and selection, and sampling multiple solutions can add overhead for extremely simple queries. Progressive sampling is identified as future work.

  • CoFiCot’s effectiveness depends on the external models that guide refinement and selection.
  • For extremely simple queries, generating k samples adds computational overhead compared with greedy decoding.
  • Progressive sampling is proposed as a possible way to address the overhead on extremely simple queries.

Supplementary Material

The supplementary material specifies CoFiCot’s staged execution: generate diverse solutions, classify difficulty, route Easy problems directly to aggregation, and iteratively refine harder problems with early exit.

  • Algorithm 1: Stage 0 generates a diverse initial solution ensemble, after which Stage 1 aggregates entropy, consensus quality, and predicted steps to assign difficulty.
  • Algorithm 1: Easy problems bypass refinement and proceed directly to final aggregation, minimizing computational overhead.
  • Algorithm 1: Medium and Hard problems enter iterative refinement using PRM-based context-aware correction and ORM-guided top-k selection.
  • Algorithm 1: The loop exits early when reevaluated difficulty converges to Easy, after which WeightedVoting produces the final answer.

B DETAILED IMPLEMENTATION OF COFICOT

The implementation defines confidence scaling, complexity prediction, adaptive routing, stateful correction, and weighted aggregation as the main operational components of CoFiCot.

  • Confidence and complexity: Entropy is transformed into normalized confidence C ∈[0, 1] with a scaled sigmoid using α = 2.
  • Confidence and complexity: A metacognitive prompt estimates reasoning depth Nsteps through look-ahead analysis without generating the full reasoning chain.
  • Adaptive routing: The algorithm samples R0, classifies difficulty, and routes Easy cases directly to WeightedVoting while refining other cases iteratively.
  • Sensitivity analysis: Uniform metric weights produce less than ±0.5% accuracy variance on MATH, while α values from 1.5 to 2.5 yield statistically similar classification boundaries.
  • Adaptive routing: Difficulty synthesis maps Easy, Medium, and Hard labels to ranks, averages them with uniform weights, and partitions the resulting [1, 3] score for routing.
  • Stateful correction: Stateful correction treats verified history as a fixed prefix and produces a rectified successor state that initiates subsequent autoregressive decoding.

C ANALYSIS

CoFiCot’s analysis weighs sampling overhead against latency and energy, emphasizing parallel generation for easy queries and the risks of greedy decoding. It also reports that ensemble consistency helps filter deceptively confident errors.

  • Latency and Compute: Generating k = 40 samples is parallelizable, keeping Easy-query latency comparable to standard parallel sampling despite higher total compute.The initial ensemble has O(1) time complexity relative to batch size when sufficient VRAM is available, whereas sequential refinement is O(N).
  • Latency and Compute: k = 40 consumes more energy than k = 1 greedy decoding, but greedy decoding risks hallucination on deceptively simple questions.The passage motivates progressive sampling as a resource-constrained alternative.
  • Latency and Compute: The ensemble-based consistency check is crucial for filtering confident errors that simple greedy decoding may retain.A progressive strategy can begin with k = 5 and scale to k = 40 when needed.

D ABLATION EXPERIMENTS

Ablations show that balanced difficulty aggregation and complementary reward-model roles are central to CoFiCot’s performance. The best results require both cautious routing and specialized correction and selection signals.

  • Difficulty Aggregation: 91.8% on GSM8K and 47.9% on MATH are achieved by balanced averaging of three difficulty labels.Pessimistic aggregation is a close second, while Democratic and Optimistic strategies more often misclassify hard problems as easy.
  • Difficulty Aggregation: Democratic and Optimistic aggregation perform worst because they are more likely to route hard problems past the refinement stage.The result supports erring toward more frequent refinement when difficulty signals disagree.
  • Reward-Model Ablation: The full PRM-and-ORM model achieves the highest accuracy, confirming that both reward models are necessary in the refinement design.The PRM localizes step-level errors for correction, while the ORM evaluates holistic solution quality for selection.
  • Reward-Model Ablation: ORM-only performance suffers because unguided self-feedback is less precise for error localization, while PRM-only performance suffers from using step scores as a proxy for complete-solution quality.Each ablation removes one specialized function from the dual reward-model design.

E REFINEMENT IMPLEMENTATION DETAILS

CoFiCot implements differentiated inference for easy, iterative, and reward-model-dependent settings. Its design uses weighted aggregation for easy problems, dynamic stopping for refinement, and adaptable verification when specialized PRMs are unavailable.

  • Easy Problems: Easy problems use Weighted Self-Consistency, selecting the final answer by ORM-weighted voting without an iterative refinement loop.This assumes the initial ensemble contains sufficient signal while retaining robust aggregation.
  • Dynamic Termination: Stage 2 terminates when solutions are reclassified as Easy or when the predefined iteration limit is reached.Reclassification indicates a high-confidence, high-consensus state.
  • Reward-Model Dependence: CoFiCot does not strictly require a domain-specific PRM, although its primary mathematical experiments use Math-Shepherd-7B.The framework is designed to remain modular across domains.
  • Reward-Model Dependence: For ARC, a general-purpose LLM verifier supplies step-level feedback when specialized PRM supervision is scarce.In this configuration, the ORM contributes more heavily to performance.
  • Potential Extensions: The adaptive coarse-to-fine philosophy is presented as potentially applicable to high-stakes multimodal domains such as biomedical informatics.This is framed as future potential rather than a demonstrated evaluation result.
Loading 2603.08251v1…