Source-linked AI summary
Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Data
Syeda Nahida Akter, Shrimai Prabhumoye, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Yejin Choi, Bryan Catanzaro
TL;DR
Evidence about reasoning data in pretraining and its interaction with post-training remains limited, despite the emphasis on reasoning-focused post-training. The paper systematically varies data properties and training phase, finding that reasoning-pretrained models retain an advantage after SFT, with a phase-dependent role for diversity and quality.
Problem
The role of reasoning data in pretraining and its interaction with post-training remain less explored than reasoning-focused mid- or post-training.
Source: §1 · p. 1Method
The study varies reasoning-data scale, diversity, and quality across pretraining and SFT, then evaluates models through downstream tasks and post-training.
Source: §2 · p. 3, §2.3 · p. 4Results
Reasoning-pretrained models followed by SFT outperform baseline models followed by SFT by 9.3% on average.
Source: §4 · p. 7Takeaways & Limitations
The findings support allocating diverse reasoning data to pretraining and high-quality data to SFT, while recognizing that high-quality pretraining data can show benefits only after SFT.
Source: §1 · p. 2, §7 · p. 10
Abstract
from arXiv · showhide
The prevailing paradigm for enhancing the reasoning abilities of LLMs revolves around post-training on high-quality, reasoning-intensive data. While emerging literature suggests that reasoning data is increasingly incorporated also during the mid-training stage-a practice that is relatively more proprietary and less openly characterized-the role of such data in pretraining remains unclear. In particular, due to the opaqueness of pretraining corpora in most frontier models, the effect of reasoning data introduced at different phases of pre- and/or post-training is relatively less reported in the scientific literature. This raises several important questions: Is adding reasoning data earlier during pretraining any better than introducing it during post-training? Could earlier inclusion risk overfitting and harm generalization, or instead establish durable foundations that later fine-tuning cannot recover? We conduct the first systematic study of how reasoning data-varying in scale, diversity, and quality-affects LLM performance when introduced at different stages of training. We find that front-loading reasoning data into pretraining is critical (19% avg gain), establishing foundational capabilities that cannot be fully replicated by later-stage SFT, even with more data. We uncover an asymmetric principle for optimal data allocation: pretraining benefits most from broad diversity in reasoning patterns (11% avg gain), while SFT is more sensitive to data quality (15% avg gain). We show that high-quality pretraining data has latent effects, activated only after SFT, and that naively scaling SFT data can be detrimental, washing away the benefits of early reasoning injection. Our results challenge the conventional separation of language modeling and reasoning, providing a principled guide for strategically allocating data across the entire training pipeline to build more capable models.
1. Introduction
The paper studies when reasoning data should enter LLM training, addressing limited evidence about its role in pretraining and its interaction with post-training. It reports durable benefits from early reasoning-data injection and an asymmetric allocation strategy across pretraining and SFT.
1. Introduction
- The study addresses limited evidence about reasoning data in pretraining and asks whether early exposure, data diversity, and data quality shape later reasoning performance.Prior research had emphasized mid- or post-training, while pretraining’s role remained less explored.Source: §1 · pp. 1–2
- +19% lead on expert-level benchmarks follows from front-loading reasoning data into pretraining, which the authors say creates a durable, compounding advantage.They argue later SFT cannot fully compensate for a weaker pretraining foundation.Source: §1 · p. 2
- Pretraining gains most from diverse data (+11%), while SFT is more sensitive to high-quality data (+15%).The authors present this as a phase-dependent allocation principle rather than a simple more-data strategy.Source: §1 · p. 2
- Blindly scaling mixed-quality SFT data yields no average improvement and reduces mathematical reasoning by 5% on average.A marginal 0.4% addition of high-quality data consistently improved performance.Source: §1 · p. 2
- High-quality data added to a diverse pretraining mix yields an additional +4% over diverse, mixed-quality pretraining after SFT.Its immediate pretraining benefit was minimal, suggesting its contribution emerges during alignment.Source: §1 · p. 3
2. Methodology
The methodology frames reasoning-data allocation between pretraining and SFT as a way to optimize downstream accuracy. It varies data properties and training stages in a controlled study, then examines whether early gains persist through reinforcement learning.
2. Methodology
- The study frames reasoning-data allocation across pretraining and SFT as an optimization of final downstream accuracy under a fixed reasoning-data budget.The objective covers math, science, code, and general reasoning tasks.Source: §2 · p. 3
2.1. Model Architecture and Baseline
- The 8B hybrid-transformer baseline combines Mamba 2, self-attention, and FFN layers and is pretrained from scratch for 1 trillion tokens.The authors state that this size balances computational feasibility with capacity for complex reasoning patterns.Source: §2.1 · p. 3
2.2. Data Pipeline
- Four reasoning datasets vary scale, diversity, quality, and complexity, including a 336-billion-token diverse corpus, curated high-quality traces, their union, and an answer-length-filtered subset.The filtered subset retains examples with answers longer than 4096 tokens.Source: §2.2 · p. 4
2.3. Synergy between Pretraining and SFT
- The study applies GRPO with verifiable rewards to assess whether reasoning gains persist through reinforcement learning.It uses Nemotron-Crossthink data for this stage.Source: §2.3 · pp. 4–5
- The study trains models from scratch for 1T tokens using 80% general pretraining data and 20% reasoning data, with variants differing in reasoning-data exposure.A baseline has no reasoning data; the other variants use large-scale diverse, small-scale high-quality, or combined reasoning data.Source: §2.3 · p. 4
- A fully crossed SFT design tests catch-up, pretraining diversity, SFT data quality and complexity, and the marginal value of more SFT data.The setup finetunes pretrained variants on differing reasoning corpora and compares common high-quality SFT across pretraining conditions.Source: §2.3 · pp. 4–5
3. Experimental Setup
The experiments pretrain and finetune 8B models, then apply reinforcement learning, using distinct training settings for each stage. Evaluation covers general-purpose reasoning and specialized math, science, code, and instruction-following benchmarks.
3.1. Training
- The 8B models are pretrained to 1T tokens, finetuned on 4.8M reasoning samples, and then trained for one GRPO epoch on Nemotron-Crossthink data.Pretraining uses 512 H100 GPUs; SFT uses a 32k context length, and GRPO uses an 8k context length.Source: §3.1 · p. 5
3.2. Evaluation Metrics
- Base-model evaluation covers zero-shot general-purpose reasoning, math, science, and code, including ARC, GSM8K, MMLU, and HumanEval benchmarks.The code evaluation reports avg@32 and pass@1 from 32 generations per prompt.Source: §3.2 · p. 6
- SFT evaluation adds AIME24 and AIME25 for math, GPQA-Diamond for science, LiveCodeBench for code, and IFEval for instruction following.The reported metric is Pass@1, averaged across repeated runs.Source: §3.2 · p. 6
- The reinforcement-learning evaluation uses complex reasoning tasks including AIME24/25, MATH-500, GSM8K, MMLU, MMLU-Pro, GPQA-Diamond, and LiveCodeBench.It follows the SFT-phase evaluation metric.Source: §3.2 · p. 6
4. Experiments and Results
Reasoning data in pretraining improves base-model performance, and its advantage persists through SFT and reinforcement learning. The final RL comparison shows an 18.57% average lead over the baseline, with a 39.32% improvement on AIME.
Immediate Foundational Gains from Reasoning Data in Pretraining
- At the 1T-token pretraining stage, ℳLDQ achieves the highest average accuracy, with gains of 28.4% in mathematics and 9% in code over baseline.The authors report that scale and diversity are more important than curated quality for establishing a broad reasoning foundation at this stage.Source: §4 · p. 6, Table 1 · p. 7
Pretraining Advantage is Maintained and Amplified Post-SFT
- 9.3% average advantage for reasoning-pretrained models over baseline models after SFT, showing that SFT does not close the pretraining gap.The authors also report that reasoning pretraining does not overfit the model and that its critical-thinking ability comes fully into potential after post-training.Source: Table 2 · p. 7, §4 · p. 7
Pretraining Strategy Dictates Final Accuracy on Expert-Level Tasks
- 18.57% average lead over the baseline after RL, with a 39.32% improvement on AIME competition math problems.The comparison uses fully aligned ℳLMQ models and ℳbase as the baseline.Source: Table 3 · p. 7, §4 · pp. 7–8
Impact of Diverse Pretraining Data on SFT Phase
- Doubling SFT data for the baseline does not catch up to reasoning-pretrained models, while the latent advantage of mixed-quality pretraining emerges after SFT.The mixed-quality model ℳLMQ becomes the top performer in the Table 4 comparison.Source: Table 4 · p. 8
5. Ablations
The ablations show that pretraining benefits from larger, more diverse reasoning data, while SFT is more effective with high-quality reasoning data. More SFT data does not reliably compensate for weaker pretraining or mixed-quality supervision.
Does the scale and diversity of the reasoning data matter in Pretraining?
- +9.09% average gain: large, diverse reasoning data outperforms a smaller, less diverse corpus in pretraining, especially on math, science, and code.Adding high-quality but less diverse data to the large, diverse mix provides minimal further benefit on reasoning tasks.Source: §5 · p. 8
The Pretraining Advantage Persists and Resists “Catch-Up” Attempts via SFT
- Doubling SFT data raises the baseline’s average score by 7.39%, but it still trails the weakest reasoning-pretrained model, which gains 3.32%.The comparison tests whether intensive SFT can make up for pretraining without reasoning data.Source: §5 · p. 8
Post-SFT, high-quality data reveals latent value
- +4.25% improvement over the diverse-data model appears after SFT for the model pretrained with high-quality but less diverse data.That advantage was muted immediately after pretraining, suggesting its value emerges during SFT.Source: §5 · p. 8
SFT is dominated by data quality, not diversity
- Large, mixed-quality reasoning datasets in SFT underperform the smaller, high-quality long-CoT dataset and can fall below the SFT baseline on math, code, and science.This contrasts with pretraining, where broad reasoning-data diversity is beneficial; SFT instead favors high-quality, reasoning-heavy data.Source: §5 · pp. 8–9, Table 5 · p. 9
How should we expand reasoning data during SFT?
- Doubling mixed-quality SFT data yields negligible average improvement and a 4.92% math-accuracy drop, while adding 0.4% more high-quality data improves average accuracy.The reported gains from targeted high-quality expansion are strongest in math and instruction-following tasks.Source: §5 · p. 9, Table 6 · p. 9
6. Related Work
Prior work examines instruction pretraining, alignment between pretraining and SFT, and reasoning-focused mid-training, but leaves open how reasoning-specific supervision at different stages contributes to durable downstream gains. This study varies reasoning data across pretraining and SFT to investigate that allocation.
Reasoning in Pretraining and Midtraining
- Cheng et al. (2024) report gains from instruction pretraining on broad reasoning tasks, but their setup does not target reasoning-intensive domains or test persistence after reasoning-heavy SFT and reinforcement learning.This paper varies reasoning-data complexity, quantity, and diversity across pretraining and SFT.Source: §6 · pp. 9–10
- Liang et al. (2025) align the SFT pool with pretraining data, while Wang et al. (2025) and AI et al. (2025) study small, high-quality reasoning mid-training datasets, especially for mathematics.The mid-training studies leave scale, complexity, and domain diversity difficult to disentangle, with generalization to science and code unclear.Source: §6 · p. 10
- Gandhi et al. (2025) find benefits from generated reasoning traces during mid-training, but their small datasets and narrow tasks leave scalability, diversity, and phase-specific allocation open.The present work studies reasoning data systematically across pretraining and SFT.Source: §6 · p. 10
7. Conclusion
The paper concludes that reasoning data should be allocated asymmetrically across training: broad diversity in pretraining, high quality in SFT, with pretraining quality able to yield benefits that emerge after SFT.
7. Conclusion
- The study finds that reasoning data should be introduced early: diverse pretraining establishes foundations that post-training alone cannot recover, while SFT benefits most from high-quality data.High-quality pretraining data can have latent benefits activated during SFT, whereas naive scaling of noisy SFT data can be harmful.Source: §7 · p. 10
C. Additional Ablations
Additional ablations find that selecting longer reasoning traces can improve reasoning-task performance, and that reusing high-quality data across pretraining and SFT is beneficial rather than overwriting earlier learning.
Anatomy of high-quality reasoning data in SFT
- The 50-times-smaller, length-filtered reasoning dataset raises the overall score to 9.87%, with stronger math, science, and code results but slightly lower instruction-following accuracy.The high-quality corpus has traces exceeding 10k tokens on average, compared with about 550 tokens in the shorter, noisier corpus; the filtered dataset is about 2% of its source.Source: C. Additional Ablations · p. 14, Table 9 · p. 15
Data Redundancy Reinforces Foundational Skills, Not Overfitting
- Using the same high-quality data in pretraining and SFT outperforms exposure during SFT alone; the authors hypothesize that SFT reinforces reasoning patterns integrated into core representations during pretraining.They further suggest that diverse pretraining helps the model capitalize on the repeated high-quality signal.Source: C. Additional Ablations · p. 15, Figure 2 · p. 15