PaperlayerView exact revision on arXiv

Source-linked AI summary

CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modes

Yufan Wu, Yinghui He, Zhengyi Hu, Lang Wei, Ruichen Li, Qifan Yang, Ting Zhu

arXiv:2608.27455v1cs.CL

TL;DR

Inference-time scaling can improve LLM reasoning but often incurs substantial generation or verification cost. CritICL transfers structured failure modes from weaker models into critique-based in-context examples through dynamic or static retrieval. It consistently outperforms standard in-context learning and matches or exceeds test-time scaling methods while using fewer generations and fewer tokens.

  • Problem

    Existing inference-time guidance methods can add overhead and may not exploit the systematic failure patterns shared across model scales.

  • Method

    CritICL builds a failure-aware critique bank from weaker models and uses dynamic, input-specific or static, family-level failure-mode retrieval to guide stronger models.

  • Results

    CritICL consistently outperforms standard in-context learning and matches or exceeds test-time scaling methods with significantly fewer generations and lower token cost.

  • Takeaways & Limitations

    Structured failure modes from weaker same-family models can provide transferable inference-time guidance for stronger models.

  • Takeaways & Limitations

    The approach assumes that models within the same family exhibit similar failure-mode distributions.

Abstract

from arXiv · show

Recent advances in inference-time scaling have significantly improved the reasoning performance of large language models (LLMs). However, these methods typically rely on repeated generation or external verification. To address this limitation, we introduce CritICL, a novel inference-time framework that improves reasoning while maintaining high efficiency. Our key insight is that LLM failure modes exhibit structured patterns across model scales within the same family. Instead of treating failures as undesirable outputs, CritICL leverages them as a source of guidance. Specifically, we utilize failure modes derived from weaker models and incorporate them into inference through critique-based in-context examples. We propose two variants: CritICL-dynamic, which adaptively predicts input-specific failure modes and retrieves critiques, and CritICL-static, which uses a global failure mode profile to provide stable guidance. Experimental results show that CritICL consistently outperforms standard in-context learning and achieves performance competitive with or superior to test-time scaling methods, while requiring significantly fewer generations and lower token cost. Code available at: https://github.com/umwyf/CRITICL

1 Introduction

Inference-time scaling improves LLM reasoning but often requires costly repeated generation or verification. CritICL instead transfers structured failure modes from weaker models through critique-based examples, with dynamic and static retrieval variants that improve performance efficiently.

  • Inference-time scaling methods improve reasoning but often require multiple generations from the same or a stronger model.
  • Weaker and stronger models within the same family exhibit consistent, transferable failure-mode distributions despite substantial capability differences.
  • CritICL transfers structured failure modes from weaker models to provide actionable guidance for stronger models during inference.
  • CritBank stores incorrect responses, failure-mode labels, and natural-language critiques for reuse as critique-based in-context examples.
  • CritICL-dynamic predicts input-specific failure modes, whereas CritICL-static uses a global model-family failure-mode profile to retrieve critiques.
  • Across mathematical benchmarks, CritICL outperforms standard in-context learning and matches or exceeds test-time scaling methods with fewer generations and lower token cost.
  • CritICL also generalizes beyond mathematical reasoning to other domains.

2 Design of CritBank and CritICL

CritICL constructs a failure-aware critique bank from weaker-model errors, then retrieves targeted examples for stronger-model inference. Its dynamic strategy predicts query-specific failure modes, while its static strategy uses family-level failure profiles.

  • Design Assumption: The method relies on the assumption that models within a family share similar failure-mode distributions, allowing weak-to-strong transfer.
  • Final Dataset: CritBank maps each question-response pair to a structured critique and a set of failure-mode labels, enabling retrieval of informative in-context samples.
  • Response Generation: Models generate chain-of-thought responses containing intermediate reasoning steps and final answers for each question.
  • Construction of CritBank: A correctness function partitions responses into correct and incorrect subsets, with incorrect responses exposing underlying reasoning failures.
  • Failure Mode and Critique Generation: For incorrect responses, a frontier LLM generates candidate failure-mode labels and natural-language critiques, then similar labels are clustered into representative modes.
  • CritICL Inference: For a new question, CritICL-dynamic predicts likely failure modes and retrieves corresponding examples from CritBank.
  • CritICL Inference: CritICL-static aggregates weaker-model failure distributions within a family to identify dominant modes and retrieve associated critiques.

3 Experiment

Experiments evaluate CritICL across Qwen models and mathematical reasoning benchmarks against standard in-context learning and test-time scaling baselines. Both variants achieve strong accuracy while reducing repeated inference and overall token usage.

  • CritBank is constructed from 15k GSM8K and MATH training questions and weaker Qwen and Llama models support evaluation on larger same-family models.
  • CritICL-dynamic and CritICL-static consistently outperform standard in-context learning baselines on Qwen family models.
  • 49.8% Pass@1 accuracy: CritICL-static surpasses Consistency@7 by 0.3 points on Qwen2.5-32B-Instruct without repeated inference.
  • 59.2% accuracy: CritICL-static outperforms all baselines, including Consistency@5 at 59.0%, on Qwen2.5-72B-Instruct.
  • 296 vs. 308 output tokens: CritICL produces comparable or fewer outputs than standard ICL despite longer inputs.
  • 3768–3897 total tokens: CritICL uses fewer tokens than test-time scaling baselines requiring 4192–7533 tokens.

4 Why CritICL Work

CritICL’s effectiveness is linked to shared, stable failure-mode distributions across model scales and to retrieval that targets persistent reasoning weaknesses. Aggregating weaker-model statistics more closely approximates large-model behavior than relying on one small model.

  • Across Qwen and Llama families, the relative ordering and magnitude of frequent failure modes remain largely stable as model size increases.
  • Aggregating three weaker models produces a failure distribution closer to the target large model than any single weaker model alone.
  • CritICL retrieves examples targeting underlying failure modes rather than random, fixed, or surface-level semantic similarity.
  • CritICL-static uses a model-family-specific global profile to retrieve critiques covering frequent and persistent target-model failure modes.
  • The selected examples provide targeted corrective signals addressing likely reasoning errors.

5 Further Analysis

Further analyses test failure-mode-based example selection against random, fixed, and semantic-similarity retrieval across mathematical and nonmathematical reasoning benchmarks. Failure-aware selection performs best, with especially large gains on challenging mathematical tasks.

  • CritICL-dynamic and CritICL-static outperform random, fixed, and semantic-similarity selection across all evaluated datasets and metrics.
  • 4–6 points in accuracy: failure-mode selection gains are observed especially on AMC23 and AIME.
  • Random and fixed selection perform weakest, while semantic similarity improves over naive baselines but remains behind CritICL.
  • CritICL’s benefits extend beyond mathematics to chemistry and biology reasoning benchmarks.

6 Conclusion

The paper presents CritICL as an inference-time framework that transfers structured failure modes from weaker to stronger models through critique-based retrieval. Across mathematical and scientific benchmarks, it reports competitive or superior reasoning with lower computational overhead than repeated-generation methods.

  • CritICL transfers structured failure modes from weaker models to stronger models through dynamic and static critique-based retrieval.
  • Inference-time reasoning methods commonly use additional computation through chain-of-thought, self-consistency, refinement, critique, or verification.
  • CritICL differs from weak-to-strong generalization work by targeting inference-time improvement rather than training-time supervision.
  • The approach builds on evidence that reasoning errors form recurring categories and that identifying error locations improves correction.
  • CritICL shifts retrieval for reasoning from semantic relevance toward failure relevance.

B.1 Quantitative Analysis of Failure-Mode Consistency

Failure-mode distributions remain strongly consistent across model scales within the same family, supporting weak-to-strong retrieval while leaving the underlying mechanism unresolved. Aggregate weak-model profiles match strong models better than individual weak models.

  • Quantitative consistency: Spearman correlations reach 0.91 for Qwen and 0.88 for Llama, with aggregate weak-model profiles also achieving the lowest JS distances.The analysis compares Spearman correlation, Kendall’s τ, Top-10 overlap, and Jensen–Shannon distance.
  • Quantitative consistency: Aggregate weak-model profiles consistently match strong models better than any individual weak model.
  • Quantitative consistency: Within-family failure distributions preserve dominant modes and relative ordering more strongly than cross-family distributions.This pattern supports weak-to-strong retrieval while indicating both general and family-specific structure.
  • Interpretation and validation: The proposed explanation attributes consistency partly to shared architectural, tokenization, pretraining, and instruction-tuning choices across family scales.The authors present this as a plausible explanation rather than a complete mechanistic account.
  • Interpretation and validation: Annotation validation used 300 independently re-annotated examples, including 100 additionally labeled by humans, and found substantial agreement.Agreement was close to observed human–human agreement, suggesting the taxonomy captures stable error categories.
  • Interpretation and validation: The fine-grained taxonomy achieved the strongest overall performance, balancing failure-specific information against retrieval-pool sparsity and noise.

C.1 Source-of-Gain Ablation

Ablations indicate that CritICL’s gains depend on aligning weak-model failure modes with retrieved critiques, not merely adding retrieved context or critique-model knowledge. CritICL-static also offers a strong efficiency advantage under matched inference settings.

  • Source of gains: Dense correct-exemplar retrieval improves over standard ICL but remains substantially below full CritICL.
  • Source of gains: Generic critiques, weak incorrect responses, and shuffled-label retrieval all underperform CritICL-static.Shuffled labels specifically break the correspondence between failure modes and retrieved critiques.
  • Source of gains: The results indicate that CritICL’s improvement cannot be explained solely by extra retrieved context or offline critique-model knowledge.Failure-mode alignment is identified as an important component of the method.
  • Inference efficiency: CritICL-static achieves the highest average accuracy among evaluated methods with one target-model generation, versus five for Consistency@5 and W2S-AlignTree.
  • Inference efficiency: MATH and macro-average improvements are statistically significant at p < 0.05, whereas GSM8K, AMC23, and AIME differences are not.

D.1 Cross-Family Transfer

CritBank transfer works across Qwen and Llama families, but same-family transfer is substantially stronger. Cross-domain transfer remains useful, while domain-specific critiques perform best.

  • Cross-family transfer: Cross-family CritBank transfer improves over standard ICL and dense correct-exemplar retrieval, but same-family transfer provides substantially larger improvements.
  • Cross-family transfer: The cross-family pattern is consistent with stronger within-family than across-family agreement in failure-mode distributions.
  • Cross-domain transfer: Domain-specific CritBanks achieve the strongest performance in both mathematical-reasoning and GPQA transfer directions.
  • Cross-domain transfer: Mixed-domain CritBanks remain close to in-domain variants, while cross-domain CritBanks retain useful but weaker transfer.
  • Cross-domain transfer: Shared failures include incorrect assumptions, missing constraints, and skipped logical steps, whereas formula application and scientific-concept confusion benefit more from in-domain information.

E.1 Performance on LLaMA Family

CritICL generalizes beyond Qwen to the Llama family, outperforming baselines across in-distribution and out-of-distribution benchmarks. It achieves these gains with a single generation and is especially strong on OOD tasks.

  • Llama-family evaluation: CritICL consistently outperforms all baselines across both ID and OOD benchmarks on Llama-family evaluation.
  • Llama-family evaluation: CritICL-static reaches 53.1 overall accuracy, exceeding Consistency@5 at 51.3 while using a single generation.
  • Llama-family evaluation: CritICL’s improvements are particularly pronounced on OOD benchmarks such as AIME24 and AIME25.The authors suggest critique-based exemplars help capture transferable reasoning patterns rather than overfitting to in-distribution examples.
  • Llama-family evaluation: The results show that CritICL’s effectiveness is not tied to a specific model family.

E.2 Inference Cost Analysis

CritICL maintains competitive or superior reasoning performance while substantially reducing inference-time token consumption across model families and mathematical tasks. Its dynamic and static variants use retrieved failure-aware critiques, with static guidance avoiding query-specific prediction overhead.

  • Inference Cost Comparison: CritICL achieves substantially lower total token consumption than test-time scaling methods while maintaining competitive or superior performance across model families and task domains.
  • Cross-Domain Performance: CritICL outperforms standard in-context learning across all GPQA domains, while consistency decoding and self-reflection require multiple generations.The evaluated domains are Chemistry, Biology, Physics, and Quantum Mechanics.
  • Cross-Domain Performance: CritICL achieves the best overall GPQA performance with a single generation, and CritICL-static improves consistently across scientific domains.
  • Cost Amortization: CritBank requires offline collection and annotation of weak-model errors, but the resulting resource can be reused across test queries and corresponding target models.
  • Inference Variants: CritICL-dynamic predicts likely failure modes for each query before retrieving corresponding critique examples and generating the final answer.
  • Inference Variants: CritICL-static retrieves critiques using persistent, model-family failure patterns without an additional query-specific prediction step.

G.5 Case Study: Failure Mode-Aligned Retrieval

The case study contrasts semantic similarity retrieval with failure-mode alignment. CritICL selects critiques that match reasoning mistakes, enabling corrective guidance across different problem contexts.

  • Case Outcome: The case study reports a corrected positive difference of 1 after failure mode-aligned examples address the comparison error.
  • Semantic Retrieval: Semantic retrieval may return fraction or set problems without the specific close-valued fraction comparison error, allowing the model to repeat its mistake.
  • Failure Mode-Based Retrieval: CritICL retrieves examples with related failure patterns, such as flawed extrema comparisons or misinterpreted area differences.
  • Corrective Guidance: Accompanying critiques explain the error source and encourage more careful comparisons, including conversion to common denominators.
  • Retrieval Strategy: Failure mode-aligned retrieval selects examples according to the type of reasoning error rather than surface-level problem similarity.
Loading 2608.27455v1…