PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training

Suorong Yang, Jiaxin Wang, Jiaming Su, Chentao Yue, Furao Shen, Peng Ye, Soujanya Poria

arXiv:2610.12933v1cs.AI

TL;DR

Reasoning post-training on ever-larger corpora can raise costs without proportional performance gains. The paper uses uncertainty-guided progressive contraction to focus supervision and optimization, reporting maintained or improved reasoning performance with much less training exposure.

  • Problem

    Scaling reasoning post-training with larger corpora incurs substantial training cost without proportional performance gains, motivating identification of trajectories with greater learning value.

    Source: §1 · p. 1

  • Method

    The framework profiles process uncertainty with Monte Carlo Reasoning Trees, contracts the corpus into a teacher-informed pool, and routes SFT toward trajectories relevant to the learner’s evolving state.

    Source: §1 · p. 2

  • Results

    Across diverse reasoning benchmarks and LLM architectures, the method maintains or improves performance while using as little as 0.22% of the full-data SFT optimization budget and over 450× less training exposure.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support shifting attention from reasoning-data scale alone toward identifying and allocating learning value throughout training.

    Source: §5 · p. 9

Abstract

from arXiv · show

Large language models have achieved remarkable advances in complex reasoning. However, scaling reasoning post-training with ever-larger corpora can substantially increase training cost without proportional performance gains. In this study, we therefore ask: how far can reasoning supervision be compressed without sacrificing capability? To this end, we propose an uncertainty-guided progressive contraction framework that concentrates supervision and optimization on the reasoning trajectories most valuable for learning. Specifically, rather than treating a teacher-generated chain-of-thought (CoT) as a single realized path, we construct uncertainty-profiled reasoning supervision by building Monte Carlo reasoning trees around intermediate reasoning states to expose process-level uncertainty. This uncertainty first contracts the original corpus into a compact teacher-informed pool. A second, learner-aware contraction then performs online routing during SFT, concentrating optimization according to the student's evolving learning state. Together, the two stages progressively contract both supervision and optimization, substantially reducing the student training budget. Experiments across diverse reasoning benchmarks and LLM families show that our method uses as little as 0.22% of the optimization budget of full-data SFT, over 450x less training exposure, while maintaining or improving reasoning performance. Notably, our framework generalizes well across heterogeneous student families, black-box teachers, and weak-to-strong transfer. This indicates that it is not only a way to accelerate training but can also enhance reasoning capability.

1 INTRODUCTION

Scaling reasoning supervision increases training cost without proportional performance gains, motivating more selective use of trajectories. The paper introduces uncertainty-guided progressive contraction and reports strong performance with substantially less optimization.

1 INTRODUCTION

  • Existing approaches strengthen supervision or filter and route fixed corpora by correctness, difficulty, or quality, but the paper argues extreme efficiency requires coupling construction with downstream allocation.The proposed shift is from simply increasing trajectory volume toward identifying and allocating trajectories with greater learning value.Source: §1 · p. 1
  • Even top-5% teacher-confident CoTs include 18.1% in the global top-10% uncertainty tail, and 30.5% of correct CoTs fall in that tail.These examples motivate profiling uncertainty at intermediate reasoning states rather than relying only on terminal confidence or correctness.Source: Figure 1 · p. 2, §1 · p. 2
  • The framework uses Monte Carlo Reasoning Trees around intermediate states to profile process uncertainty, then contracts supervision and routes SFT optimization according to the learner’s evolving state.The first contraction forms a compact teacher-informed candidate pool; the second concentrates optimization on learner-relevant trajectories.Source: §1 · p. 2
  • The method uses as little as 0.22% of the full-data SFT optimization budget, over 450× less training exposure, while maintaining or improving reasoning performance.The paper presents this as a result of progressively concentrating both supervision and optimization.Source: §1 · p. 2
  • Across diverse benchmarks and LLM architectures, the framework maintains or improves reasoning performance while substantially accelerating SFT.Reported results include generalization across heterogeneous student families, black-box teachers, and weak-to-strong transfer.Source: §1 · p. 2

2 THE PROPOSED METHOD

The method contracts supervision twice: teacher-side uncertainty profiling selects a compact pool, then learner-side routing directs SFT toward trajectories less well captured by the current student.

2.1 PROBLEM FORMULATION AND OVERVIEW

  • The framework first retains verified trajectories with high teacher-side uncertainty, then routes SFT toward the M trajectories with the largest learner-side response losses.Higher response loss indicates that a teacher trajectory remains less well captured by the current student; routing scores update as the learner state evolves.Source: §2.1 · p. 3, §2.3 · pp. 4–5

2.2 UNCERTAINTY-GUIDED REASONING AND PROFILING

  • Monte Carlo continuations from intermediate reasoning states estimate correct-answer reachability and branch-outcome uncertainty, which are aggregated into a length-normalized trajectory profile.Higher trajectory uncertainty marks paths that more often pass through states whose alternatives disagree in eventual correctness.Source: §2.2 · pp. 3–4

2.3 LEARNER-AWARE DATA ROUTING

  • Below 0.74% of vanilla full-set SFT exposure—over 135× fewer sample-level exposures—the method maintains downstream reasoning performance.The teacher-side pool is used for one pass, followed by two passes over the learner-routed active set.Source: §2.3 · p. 5

3 EXPERIMENT

Across reasoning benchmarks, progressive contraction sharply reduces student SFT budgets while maintaining or improving performance. Results span heterogeneous students, weak-to-strong transfer, and API-only teachers; ablations support contributions from both uncertainty profiling and learner-aware routing.

3.1 EXPERIMENTAL SETUP

  • Experiments use Qwen3-8B as the usual teacher, diverse Qwen, Gemma, LLaMA, and Mistral students, LoRA-based SFT, and competition mathematics, general mathematics, and scientific reasoning evaluations.The setup also includes a weaker Qwen3-1.7B teacher and API-only closed-source teachers.Source: §3.1 · pp. 5–6

3.2 MAIN RESULTS

  • 0.22–1.91% of the full-SFT optimization budget suffices for Qwen3.5-4B/9B, reducing SFT time from 8.47/9.23 hours to 0.01–0.07 hours while models remain competitive with or outperforming full-set SFT.Under ρ = 1%, the paper reports over 800× wall-clock acceleration.Source: §3.2 · p. 6
  • Gemma, LLaMA, and Mistral students maintain or improve reasoning performance using approximately 0.45% of the full-SFT optimization budget under shared Qwen3-8B uncertainty profiling.The figure caption describes the cumulative budget reduction as over 99.6% versus full-set SFT.Source: Figure 2 · p. 6, §3.2 · p. 6
  • With a Qwen3-1.7B teacher, Qwen3.5-4B and Qwen3-14B gain up to 14.4% at ρ = 5% with at most 0.1 GPU hours of student training.The reported interpretation is that uncertainty-guided profiling can expose locally informative trajectories while learner-aware routing selects those useful to the stronger student.Source: §3.2 · pp. 6–7, Table 2 · p. 7
  • With API-only DeepSeek-V4-Pro and Kimi-K2.6 teachers, the method improves over vanilla distillation by up to 13.4% without changing the behavior-based uncertainty estimator.These experiments use the first 1,000 Nemotron-Math samples.Source: §3.2 · p. 7, Figure 3 · p. 8
  • The compact 9B student remains highly competitive with reasoning models one to two orders of magnitude larger across the reported benchmarks.Table 3 compares larger reasoning models across various reasoning benchmarks.Source: §3.2 · p. 7, Table 3 · p. 7

3.3 MECHANISM ANALYSIS AND ABLATIONS

  • At a matched 1% budget, uncertainty-guided supervision outperforms vanilla distillation by 3.6% and 4.0% on Qwen3.5-4B and 9B, respectively.Separately, the method surpasses full SFT in 150 steps, compared with about 1,500 steps; both contraction stages improve accuracy and their combination performs best.Source: §3.3 · pp. 7–9, Table 5 · p. 8
  • Among verified-correct trajectories and without learner-aware routing, uncertainty-based selection improves average performance by 1.6%–4.5% over matched-size random selection.This comparison indicates that final-answer correctness alone does not fully characterize supervision value.Source: §3.3 · p. 8, Table 4 · p. 8

4 RELATED WORK

Prior work studies uncertainty as a signal of LLM reliability and adapts reasoning supervision and training-data selection to improve learning value. These strands motivate structured uncertainty signals and data routing.

Uncertainty in LLM Reasoning and Learning

Data-Efficient Distillation and Adaptive Data Routing

  • Reasoning distillation increasingly transfers intermediate rationales or long-form CoTs, while evidence suggests effectiveness depends on trajectory form and learner compatibility [Liu et al., 2026; Wu et al., 2026a;b; Yang et al., 2026b; Zhang et al., 2026b].Prior adaptations include trajectory filtering, prefix alignment, difficulty-aware supervision, and student-conditioned selection.Source: §4 · p. 9
  • Data-routing methods allocate limited optimization toward useful examples using quality, diversity, gradient or influence, and model-dependent learning-dynamics signals [Wang et al., 2024; Zheng et al., 2023; Yang et al., 2022; Engstrom et al., 2024].Examples include DEITA (Liu et al., 2024), LESS (Xia et al., 2024), Data Agent (Yang et al., 2026a), and RHO-Loss (Mindermann et al., 2022).Source: §4 · p. 9

5 CONCLUSION

The conclusion presents progressive contraction as a way to concentrate learning value in reasoning supervision and improve data efficiency. Across benchmarks and architectures, the authors report strong generalization and performance maintained without sacrificing capability.

5 CONCLUSION

  • The framework exposes and concentrates learning value in reasoning supervision, accelerating post-training without sacrificing capability.The authors describe it as a practical tool for reducing training burden under limited computational resources.Source: §5 · p. 9
  • Across diverse benchmarks and architectures, the method improves data efficiency with strong generalization.Source: §5 · p. 9
  • The results support shifting attention from reasoning-data scale alone to identifying and allocating learning value throughout training.Source: §5 · p. 9

Related papers

Loading 2610.12933v1…