PaperlayerView exact revision on arXiv

Source-linked AI summary

Scale or Reason? A Compute-Equivalent Analysis of Reasoning Distillation

Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Kevin El Haddad, Céline Hudelot, Pierre Colombo

arXiv:2509.22193v3cs.CL

TL;DR

Reasoning distillation is widely used, but its 5-20× longer traces create a compute-allocation trade-off against training a larger IFT model. The paper isolates supervision format with paired teacher outputs and evaluates students from 0.5B to 14B on 18 benchmarks. At matched FLOPs, IFT is usually on or near the Pareto frontier, while reasoning helps chiefly on open-ended tasks at 7B and above; using 25-50% reasoning data captures most of that benefit at lower cost.

  • Problem

    Reasoning distillation’s 5-20× longer traces make its compute trade-off against scaling a larger IFT model an open research question.

    Source: §1 · p. 1

  • Method

    A single teacher generates paired IFT and reasoning outputs for identical prompts by toggling reasoning mode, while students span 0.5B to 14B and 18 benchmarks.

    Source: §1 · p. 2, §2.1 · p. 2

  • Results

    At matched FLOPs, IFT lies on or near the Pareto frontier across most configurations, whereas reasoning reaches it mainly on open-ended tasks at 7B parameters and above.

    Source: §1 · p. 2, §7 · p. 8

  • Takeaways & Limitations

    A sequential curriculum using 25-50% reasoning data recovers most of the accuracy benefit at a fraction of the training cost while mixed training below 50% preserves IFT inference efficiency.

    Source: §1 · p. 2, §7 · p. 9

    The study is limited to models up to 14B and the distillation paradigm, so further scaling and dedicated reinforcement-learning studies are needed to test whether the conclusions generalize.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Distilling reasoning traces from strong teacher models has become the standard recipe for building capable small language models. Yet reasoning traces are 5-20$\times$ longer than standard instruction fine-tuning (IFT) outputs, meaning every practitioner who chooses reasoning distillation implicitly forgoes training a larger IFT model on the same compute budget. Whether this trade-off is worthwhile remains unaddressed. We study it with a controlled experiment: a single teacher generates paired IFT and reasoning outputs for identical prompts by toggling only its reasoning mode, isolating supervision format as the sole variable. Training students at five scales (0.5B to 14B) and evaluating on 18 benchmarks, we find that at matched FLOPs, IFT lies on or near the Pareto frontier across the majority of configurations. Reasoning reaches the Pareto frontier only on open-ended tasks at 7B and above. Even there, a sequential curriculum mixing just 25-50\% reasoning data with IFT captures most of the accuracy benefit at far lower compute cost.

1 Introduction

The paper asks whether reasoning distillation justifies its higher compute cost than scaling a larger IFT model. In a controlled multi-scale evaluation, IFT is usually compute-efficient, while reasoning helps mainly for open-ended tasks at sufficient capacity and can be approximated with partial reasoning data.

1 Introduction

  • Reasoning traces are 5-20× longer than standard IFT outputs, so reasoning supervision consumes proportionally more compute and competes with training a larger IFT model under a fixed budget.The study targets a compute-allocation question left open by prior fixed-model-size evaluations.Source: §1 · p. 1, Figure 1 · p. 1
  • At matched training and inference FLOPs, IFT models lie on or near the Pareto frontier across most configurations.Reasoning approaches the frontier mainly on open-ended tasks and at larger scales.Source: Figure 1 · p. 1, §1 · p. 2
  • Reasoning benefits depend jointly on output format and capacity: open-ended tasks benefit consistently, whereas multiple-choice tasks generally do not, and the frontier is reached only at 7B parameters or above.The paper identifies format as the primary predictor and capacity as a secondary prerequisite.Source: §1 · p. 2
  • A sequential curriculum with 25-50% reasoning data captures most of reasoning’s accuracy benefit at a fraction of the training cost.Mixed training below the 50% threshold also improves accuracy while preserving efficient IFT inference costs.Source: §1 · p. 2
  • The study provides a systematic answer to the compute-allocation question in reasoning distillation.Its comparison isolates supervision format while evaluating the resulting trade-off across configurations.Source: §1 · p. 2

2 Experimental Setup

The experiment compares IFT and reasoning supervision under a paired teacher design, then evaluates students across scales, training curricula, task families, and compute budgets. Evaluation uses 18 benchmarks and tracks the extra FLOPs caused by longer reasoning traces.

2.1 Distillation

  • The paired design uses one teacher and identical prompts, toggling reasoning mode so supervision format is the sole independent variable.IFT uses question-answer pairs, whereas reasoning uses question-trace-answer tuples.Source: §2.1 · p. 2
  • Students are Qwen2.5 base models at five scales from 0.5B to 14B, trained on general-purpose and reasoning-centric corpora.The student family differs from the teacher family to limit shared pretraining overlap.Source: §2.1 · p. 2
  • Sequential training varies the reasoning proportion across 0%, 25%, 50%, 75%, and 100%, with IFT examples preceding reasoning examples; shuffled mixed training is also evaluated.All models train for one epoch on 16 H100 GPUs with a fixed global batch size of 32.Source: §2.1 · p. 2

2.2 Evaluation

  • Evaluation covers 18 benchmarks organized by domain and answer format into General-MC, General-OE, Math-MC, and Math-OE families.The taxonomy separates general versus math domains and multiple-choice versus open-ended outputs.Source: §2.2 · p. 2
  • Answers are judged by three models that extract final answers before assessing semantic equivalence, so scores reflect correctness rather than reasoning-trace quality.Median absolute error across judges is 1.2 points, with pairwise agreement below 2 points on all benchmarks.Source: §2.2 · pp. 2–3
  • Reasoning’s 5-20× longer traces create proportional training and inference FLOP overhead, which the study tracks for compute-matched comparisons.The overhead applies to both forward and backward passes.Source: §2.2 · p. 3

3 Is Reasoning Worth the Compute Cost?

At fixed parameter counts, reasoning supervision often improves accuracy, but compute-matched comparisons reverse this advantage: IFT generally defines the efficiency frontier, with exceptions emerging for larger models and open-ended tasks.

3 Is Reasoning Worth the Compute Cost?

  • The study compares equal-size, matched-training-compute, and matched-inference-compute models using general-purpose and math-centric corpora.The figures plot accuracy against training and inference FLOPs for the two supervision formats.Source: §3 · p. 3, §3.1 · p. 3, §3.2 · p. 4

3.1 At Fixed Size, Reasoning Helps

  • At equal model size, reasoning supervision outperforms IFT on general-purpose training, with a 3B reasoning model exceeding 14B IFT on Math-OE by 1.7 points.On General-OE, the 3B reasoning model comes within 1.4 points of the 14B IFT baseline, while General-MC is an exception at the smallest scales.Source: §3.1 · p. 3
  • Math-centric reasoning surpasses general-purpose reasoning on math tasks from 3B upward, but its general-domain penalty is tied to reasoning format rather than the specialized data.At 14B, Math-OE is 69.2 versus 74.2 for general-purpose reasoning, whereas same-corpus IFT remains competitive on general tasks.Source: §3.1 · pp. 3–4

3.2 At Fixed FLOPs, IFT Dominates

  • At matched training and inference FLOPs, IFT models define or closely approach the Pareto frontier across most evaluations, while reasoning reaches it mainly on open-ended tasks and larger scales.The training-compute gap may narrow beyond 14B parameters, but reasoning outputs remain substantially more expensive during inference.Source: §3.2 · pp. 4–5, Figure 3 · p. 5
  • A 7B IFT model reaches accuracy similar to a 3B reasoning model on general multiple-choice tasks with 5× less training compute.On open-ended math, 7B IFT similarly matches 1.5B reasoning at 1.6× lower training cost.Source: §3.2 · p. 4
  • Reasoning outputs are 7-15× longer than same-size IFT outputs, producing higher inference costs at every scale and a roughly 10-15× overhead on multiple-choice tasks.On open-ended tasks, the overhead is approximately 7×, yet reasoning can approach or join the Pareto frontier.Source: §3.2 · p. 5

4 What Drives Reasoning Efficiency?

Reasoning efficiency is governed more by output format than knowledge domain, while model capacity determines whether its longer traces become worthwhile. Open-ended tasks and models at or above 7B form the main beneficial regime.

4.1 Format Predicts Value; Domain Does Not

  • Holding domain constant while varying format tests whether efficiency depends on knowledge complexity or output structure.The comparison spans law, code, and physics, each with open-ended and multiple-choice benchmarks.Source: §4.1 · p. 5
  • Open-ended tasks place reasoning models on or near the Pareto frontier, whereas multiple-choice tasks retain a substantial efficiency gap despite similar knowledge domains.On C_Code_Summarization, 7B reasoning matches IFT at equal inference cost with only 1.3× training overhead; mmlu_pro_code shows a 10× inference gap at comparable performance.Source: §4.1 · p. 5

4.2 Model Capacity Determines Whether Traces Help or Hurt

  • Reasoning captures both compute frontiers from 7B upward, matching or exceeding IFT on open-ended tasks and surpassing it in multiple-choice accuracy at higher inference cost.This capacity threshold makes reasoning dominant on open-ended tasks beyond 7B, while multiple-choice gains remain less compute-efficient.Source: §4.2 · p. 6
  • Across scales and task families, incorrect reasoning traces are 1.5 to 2× longer than correct traces, indicating that longer outputs correlate with failure rather than deeper processing.The failure-to-correct length ratio shrinks from roughly 2× at 0.5B to 1.5× at 14B.Source: §4.2 · p. 6
  • Smaller reasoning models have poor accuracy-to-inference-cost ratios because long incorrect traces skew their average generation length.Length-based early stopping reduces inference FLOPs but loses corresponding accuracy without shifting the Pareto frontier.Source: §4.2 · p. 6
  • The beneficial regime is jointly defined by format and capacity: open-ended reasoning becomes viable, then dominant, as models scale beyond 7B.For open-ended deployment, the practical choice becomes smaller reasoning models for memory constraints versus larger IFT models for lower latency.Source: §4.2 · p. 6, §4.1 · p. 6

5 Leveraging IFT and Reasoning with Hybrid Data Mixtures

Hybrid data mixtures reduce the compute burden of reasoning distillation in complementary ways: sequential training lowers training cost, while mixed training can preserve concise inference. The strongest benefits occur with modest reasoning proportions, whereas reasoning-heavy mixtures trigger a costly mode switch.

Sequential training

  • 25% reasoning data captures most sequential-training gains at a fraction of full reasoning-distillation cost.The result is broadly consistent across task families and model sizes, except for the smallest models on general multiple-choice tasks.Source: §5 · p. 7, Figure 5 · p. 7
  • Sequential training reaches the Pareto frontier for open-ended tasks at 25%, 50%, and 75% reasoning proportions.These mixtures use IFT first, followed by reasoning traces, and match the efficiency of full reasoning distillation with less reasoning data.Source: §5 · p. 7

Mixed training

  • At 25–50% reasoning data, mixed training improves mathematical accuracy over IFT while maintaining short outputs and avoiding the usual inference-cost penalty.Mixed training combines both data types in a single shuffled pass rather than staging them sequentially.Source: §5 · p. 7, Figure 6 · p. 8
  • Above 50% reasoning data, mixed training switches to reasoning-style generation, eliminating its inference-cost advantage while remaining slightly less accurate than 75% sequential training.Output lengths converge toward full reasoning distillation once reasoning data becomes predominant.Source: §5 · p. 7

Practical implications

  • Sequential and mixed training address different constraints: sequential mixtures reduce training compute, whereas sub-50% mixed mixtures preserve efficient inference.Together, they show that full reasoning distillation is unnecessary when either training or inference compute is the binding constraint.Source: §5 · pp. 7–8

6 Related Work

Prior work established instruction tuning, chain-of-thought supervision, knowledge distillation, and compute-optimal scaling as foundations for this study. The paper addresses an underexplored question: when reasoning distillation is more compute-efficient than standard IFT.

Instruction tuning and reasoning

  • Instruction fine-tuning aligns language models with human instructions, while chain-of-thought supervision adds intermediate reasoning steps and improves several reasoning benchmarks.The cited literature includes arithmetic, symbolic, and commonsense reasoning settings.Source: §6 · p. 8

Knowledge distillation

  • Knowledge distillation transfers capabilities from strong teachers to smaller students, including through teacher-generated IFT or reasoning traces.Text-based distillation is positioned as a lower-cost alternative to reinforcement learning while preserving performance.Source: §6 · p. 8

Research gap

  • The structural conditions determining when reasoning distillation is more compute-efficient than standard IFT remain underexplored.This gap motivates comparing supervision formats under matched compute budgets.Source: §6 · p. 8

Compute-optimal scaling

  • Compute-optimal scaling formalizes trade-offs among model capacity, dataset size, training compute, and inference-time reasoning compute.Recent work also evaluates performance as a function of token cost for API-based models.Source: §6 · p. 8

7 Conclusion

The paper finds that standard IFT is usually the compute-efficient choice, while reasoning reaches the Pareto frontier only for open-ended tasks at sufficient model capacity. Hybrid mixtures recover much of reasoning’s benefit with lower training or inference cost.

Conclusion

  • IFT lies on or near the Pareto frontier for most configurations, whereas reasoning reaches it only for open-ended outputs and models with at least 7B parameters.Below that capacity, answer length is dictated by failure, inflating inference costs.Source: §7 · p. 8
  • Sequential training with 25–50% reasoning data recovers most accuracy gains at a fraction of the training cost.Mixed training below a 50% reasoning threshold improves accuracy while preserving standard IFT inference efficiency.Source: §7 · p. 9

Limitations

The study evaluates models only up to 14B parameters and focuses specifically on distillation, leaving broader scaling behavior and reinforcement-learning comparisons for future work.

Scope

  • The experiments cover models up to 14B parameters, so additional scaling is needed to test whether the identified conditions remain stable beyond that capacity.The 7B Pareto-frontier finding is confirmed at 14B, but larger models are not evaluated.Source: Limitations · p. 9
  • The study examines distillation rather than reinforcement learning, so dedicated reinforcement-learning experiments are needed to confirm whether the observations generalize.The authors note that reinforcement learning typically underperforms distillation and costs more at the studied scales, but treat this as a hypothesis requiring direct study.Source: Limitations · p. 9

Ethics Statement

The paper motivates compute-aware evaluation of reasoning distillation, situates distillation as important for smaller models, and documents paired data and evaluation prompts used in the study.

Ethics Statement

  • Rising accuracy-focused training and inference strategies motivate evaluating reasoning distillation against IFT under matched compute budgets.The authors emphasize weighing accuracy gains against computational and environmental costs.Source: Ethics Statement · p. 9

A Distillation Setup Positioning

  • Distillation is presented as standard for smaller models, with prior reports showing distilled models outperform RL-only counterparts, including average scores of 71.55 versus 58.45 and 60.71 versus 58.91.These comparisons position distillation as important for strong performance at smaller scales.Source: A Distillation Setup Positioning · p. 13

B Prompting Details

  • The section documents paired IFT-reasoning examples, answer-generation prompts, and LLM-judge prompts used in the study.The judge protocol extracts final answers before assessing semantic equivalence, with reported median absolute error of 1.2.Source: B Prompting Details · p. 13

C Training Hyperparameters

Training uses matched sample counts and explicit FLOPs accounting, then analyzes scaling, task coverage, stopping, and inference-cost trends across configurations.

C Training Hyperparameters

  • All runs use one epoch, identical sample counts, global batch size 32 across 16 H100 GPUs, WSD scheduling, AdamW_fused, and grid-searched peak learning rates.Table 2 lists the selected peak learning rates by student model and data format.Source: C Training Hyperparameters · p. 13, Table 2 · p. 13

D.1 Notations

  • The study defines FLOPs variables including sample count, attention heads, transformer layers, sequence length, prompt length, and generation length.These quantities support the training and inference calculations.Source: §D.1 · p. 13

D FLOPs Computation

  • Training FLOPs are computed from model architecture and sequence length, while inference FLOPs depend on architecture, prompt length, and generation length.The notation includes vocabulary size, hidden and feed-forward dimensions, layer count, prompt length, and generation length.Source: §D.2 · p. 13, §D.3 · p. 13, §D.1 · p. 13

E Details on Interpolation Curves

  • Scaling curves interpolate accuracy from FLOPs with a constrained saturating function fitted by minimizing mean absolute error.The function uses positive α, β, and γ, with 0 < δ ≤ 1 and an asymptote no higher than teacher accuracy.Source: E Details on Interpolation Curves · p. 13

F Extended Task Evaluation

H Generation Early-Stopping

  • Early stopping fails to improve the inference Pareto frontier because lower inference cost directly reduces accuracy, while some math tasks benefit from longer generation.The longer-generation analysis also indicates extrapolation beyond training-time trace lengths.Source: H Generation Early-Stopping · p. 14

I Inference Cost Scaling Trends

  • Inference-cost scaling exponents align between IFT and reasoning for General-OE, Math-MC, and Math-OE, whereas General-MC reasoning has larger scaling coefficients.The larger General-MC coefficients reflect higher cost for marginal gains.Source: I Inference Cost Scaling Trends · p. 14

J Task-Level Results

The task-level appendix provides paired-data and evaluation-prompt examples, judge reliability information, and figures breaking down ratios, costs, scaling, and task performance.

Task-Level Results

  • The appendix includes examples of paired IFT and reasoning answers, including character counts for generated data.These examples support inspection of the supervision formats and their lengths.Source: Table 3 · p. 15, Table 4 · p. 16
  • Evaluation prompts and LLM-judge prompts are documented, with judges extracting final answers before assessing semantic equivalence.Human verification and inter-judge agreement are reported alongside median absolute error and pairwise comparison errors.Source: Table 5 · p. 17, Table 6 · p. 18
  • Figures 7–9 compare reasoning ratios, inference FLOPs, and Nemotron-Gemma teacher-student performance for law, code, physics, and alternative configurations.These visuals extend the task-level analysis across domains and model setups.Source: Figure 7 · p. 19, Figure 8 · p. 19, Figure 9 · p. 20
  • Figures 10–14 examine reasoning-ratio effects, inference accuracy-cost trade-offs, early stopping, maximum generation length, and FLOPs scaling with student size.The analyses span 4B Nemotron-Gemma models, math tasks, and IFT-versus-reasoning scaling fits.Source: Figure 10 · p. 20, Figure 11 · p. 21, Figure 12 · p. 22
  • Figures 15–20 provide task-level analyses of mono-phasic models, sequential and mixed training, reasoning ratio, training and inference FLOPs, and answer length.Source: Figure 15 · p. 23, Figure 16 · p. 23, Figure 17 · p. 23

Related papers

Loading 2509.22193v3…