Source-linked AI summary
Test-Time Scaling Makes Overtraining Compute-Optimal
Nicholas Roberts, Sungjun Cho, Zhiqi Gao, Tzu-Heng Huang, Albert Wu, Gabriel Orlanski, Avi Trost, Kelly Buchanan, Aws Albarghouthi, Frederic Sala
TL;DR
Existing pretraining scaling laws do not determine how models should be trained for repeated-sampling deployment, where model size, training, and inference costs are coupled. T2 jointly models loss or pass@k across these decisions and finds inference-aware optima favor smaller, overtrained models across eight tasks, with results persisting after post-training.
Problem
Existing pretraining and test-time scaling laws address separate decisions and do not capture how model size, training, and repeated-sampling deployment should be jointly optimized.
Method
T2 jointly models performance as a function of model size N, training tokens D, and samples k, using complementary loss-based and pass@k-based approaches under end-to-end compute budgets.
Results
Across eight downstream tasks, T2 shifts the compute-optimal choice toward substantially smaller, more overtrained models than Chinchilla, with the trade-offs persisting after post-training.
Takeaways & Limitations
Inference cost should inform pretraining allocations: T2-predicted overtrained models consistently outperform Chinchilla checkpoints under fixed inference budgets.
Takeaways & Limitations
The loss-based approach relies on a Beta-distribution assumption for task difficulty, while its NLL-style objective is an upper-bounding surrogate for negative log expected pass@k.
Abstract
from arXiv · showhide
Modern LLMs scale at test-time, e.g. via repeated sampling, where inference cost grows with model size and the number of samples. This creates a trade-off that pretraining scaling laws, such as Chinchilla, do not address. We present Train-to-Test ($T^2$) scaling laws that jointly optimize model size, training tokens, and number of inference samples under fixed end-to-end budgets. $T^2$ modernizes pretraining scaling laws with pass@$k$ modeling used for test-time scaling, then jointly optimizes pretraining and test-time decisions. Forecasts from $T^2$ are robust over distinct modeling approaches: measuring joint scaling effect on the task loss and modeling impact on task accuracy. Across eight downstream tasks, we find that when accounting for inference cost, optimal pretraining decisions shift radically into the overtraining regime, well-outside of the range of standard pretraining scaling suites. We validate our results by pretraining heavily overtrained models in the optimal region that $T^2$ scaling forecasts, confirming their substantially stronger performance compared to pretraining scaling alone. Finally, as frontier LLMs are post-trained, we show that our findings survive the post-training stage, making $T^2$ scaling meaningful in modern deployments.
1 Introduction
Pretraining and test-time scaling are coupled, but existing scaling laws treat them separately. T2 jointly optimizes model size, training tokens, and repeated samples, finding that inference-aware optima favor smaller, overtrained models across tasks and persist after post-training.
- Existing scaling laws separately optimize pretraining or deployment, despite model size and training duration affecting both inference quality and cost.
- Repeated sampling creates a nonlinear trade-off: smaller models cost less per sample but are weaker per sample, while pass@k improves with multiple attempts.
- T2 jointly optimizes model size N, training tokens D, and inference samples k under budgets including training and inference compute.
- T2 combines loss-based and pass@k-based formulations, which agree closely and both recommend substantial overtraining and test-time scaling.
- Across eight tasks, accounting for repeated-sampling cost shifts the optimum toward substantially smaller and more overtrained models than Chinchilla prescribes.
- Overtrained models consistently outperform Chinchilla checkpoints, and T2-derived trade-offs persist after post-training despite greater fine-tuning difficulty.
2 Background
Chinchilla models pretraining loss from model capacity and dataset size under a compute budget, while pass@k measures success from repeated independent samples. These provide the separate foundations that T2 connects for inference-aware allocation.
- Chinchilla models pretraining loss as a function of model capacity N and dataset size D, the number of training tokens.
- The loss combines an irreducible floor with reducible contributions from model size and dataset size, whose parameters are fit empirically.
- Under a pretraining budget C_train ≈ 6ND, compute-optimal model size and training tokens scale at similar rates, with a ≈ b ≈ 0.5.
- Pass@k is the probability that at least one of k independent samples is correct, aggregated across benchmark problems.
3 Estimating Optimal Pretraining Allocations for Test-Time Scaling
T2 scaling jointly models model size, training data, and repeated test-time sampling, then optimizes these choices under separate training and inference budgets. It uses both loss-based and accuracy-based formulations, with inference correction allocating more samples to smaller models.
- Optimization objectives: T2 optimizes model size N, dataset size D, and inference samples k jointly under training and inference compute constraints.The objectives minimize loss or maximize accuracy subject to 6ND ≤ Ctrain and 2Nk ≤ Cinf.
- Approach 1: Task loss: Approach 1 extends Chinchilla loss scaling with a power-law term for repeated sampling and optimizes the resulting NLL objective.The formulation recovers standard Chinchilla scaling when k = 1 and retains the irreducible-loss limit as N, D, and k grow.
- Approach 2: Task accuracy: Approach 2 models pass@k accuracy directly by combining a sigmoid mapping from NLL to single-pass accuracy with a Beta distribution over per-question accuracies.The distributional model accounts for variation in task difficulty and avoids the naive pass@k estimate's overestimation from concavity.
- Inference cost correction: Inference correction uses Cinf ≈ 2Nk, so smaller models receive more repeated samples than larger models under the same inference budget.The corrected budget relation is substituted into both T2 approaches before optimizing pretraining decisions.
- Forecasts: Both T2 approaches forecast extreme overtraining relative to standard pretraining scaling, including substantially smaller optimal models and larger optimal training sets.Figure 2 compares optimal tokens per parameter, model sizes, and training set sizes against Hoffmann et al. (2022).
4 Experiments
Experiments test whether T2 scaling changes pretraining choices, extrapolates to overtrained checkpoints, and survives post-training. Across these evaluations, T2 favors smaller, substantially overtrained models with repeated sampling, including after post-training.
- Experimental setup: Across eight real and synthetic tasks, the experiments compare T2 predictions with Chinchilla scaling and test extrapolation to overtrained checkpoints.The evaluation uses checkpoints with fewer than 1B parameters and includes four real and four synthetic tasks.
- RQ1: Pretraining and test-time scaling: T2 improves monotonically over Chinchilla across evaluation tasks, whereas inference-corrected Chinchilla scaling is non-monotonic in Ctrain.The isoFLOP curves are macro averaged across all eight tasks.
- RQ1: Pretraining and test-time scaling: Both T2 approaches forecast dramatically smaller and more overtrained models than Chinchilla, with more repeated samples at every compute scale.Approach 2 recommends more aggressive overtraining than Approach 1, despite modeling pass@k rather than NLL.
- RQ2: Extrapolation: 2.8% relative error for Approach 1 versus 8.4% for Approach 2 when extrapolating to 16 new overtrained checkpoints.Both approaches somewhat overestimate performance, but both extrapolate to the overtraining regime.
- RQ2: Extrapolation: Best small overtrained checkpoints outperform Chinchilla-optimal checkpoints under inference correction across all eight tasks.This supports the claim that T2's overtraining finding is not merely an artifact of the fitted approaches.
- RQ3: Post-training: After post-training, the optimal frontier still shifts toward smaller overtrained models with more test-time samples across all three tasks and methods.The shift is somewhat subdued relative to base models, but overtrained checkpoints still outperform Chinchilla-optimal checkpoints.
5 Conclusion
The paper presents T2 scaling as a joint framework for choosing model size, training tokens, and repeated test-time samples. It concludes that accounting for inference compute favors smaller, longer-trained models, while identifying larger-scale validation and richer inference-cost modeling as future work.
- Conclusion: T2 scaling jointly optimizes model size, training tokens, and repeated test-time samples under fixed pretraining and inference budgets.The framework addresses the intersection of pretraining scaling laws, test-time scaling, and overtrained models.
- Conclusion: Accounting for repeated-sampling compute makes the optimal model substantially smaller and more overtrained than under standard Chinchilla scaling.This conclusion is supported across NLL-based and pass@k-based formulations and eight real and synthetic downstream tasks.
- Conclusion: T2 findings extrapolate to the predicted overtraining region and persist after post-training.The paper reports validation of both the overtraining extrapolation and post-training persistence.
- Conclusion: Future work includes validation at larger scales, transformer-specific inference-cost models, and explicit modeling of additional test-time-scaling factors.The supplied conclusion passage states these as planned extensions.
A.3 Overtraining
The paper reframes overtraining as beneficial when models are deployed with repeated-sampling inference budgets. T2 provides a framework for selecting how much to overtrain under a joint train-and-test compute allocation.
- A.3 Overtraining: Modern models often train smaller models on more tokens than Chinchilla recommends, and T2 studies this overtraining under repeated-sampling inference.Chinchilla prescribes roughly 20 training tokens per model parameter, while modern releases routinely deviate from that ratio.
- A.3 Overtraining: T2 provides a principled framework for determining how much to overtrain given a joint train-and-test compute allocation.The framework treats overtraining as actively beneficial when deployment includes a repeated-sampling inference budget.
B Per-Task Analysis
Per-task isoFLOP analyses examine T2 predictions under different inference budgets and modeling approaches. The paper reports relatively stable overtraining predictions across inference budgets for both approaches.
- Per-task analysis: IsoFLOP profiles are provided for each evaluation task under both T2 modeling approaches.The profiles appear in Figure 6 for Approach 1 and Figure 7 for Approach 2.
- Per-task analysis: Overtraining predictions remain relatively stable across inference budgets for both T2 approaches.This stability is reported across the individual task profiles.
C.1 Checkpoint Scaling Grid
The checkpoint grid combines existing pretrained checkpoints with newly trained overtrained checkpoints across model sizes and training budgets. It characterizes overtraining using tokens per parameter while retaining fixed training configurations for comparability.
- The grid spans 5M to 901M parameters and training FLOPs from 1.25 × 10^16 to 2.56 × 10^19.
- Each grid cell reports tokens per parameter, indicating the degree of overtraining for that checkpoint.
- New overtrained checkpoints were pretrained from scratch using the same fixed hyperparameters as the Chinchilla-optimal checkpoints.
D Post-Training Details
The post-training setup compares standard fine-tuning with supervised fine-tuning across three tasks and evaluates models after convergence. The supplied figure captions also identify scaling-profile and checkpoint-grid visualizations used elsewhere in the analysis.
- Post-training variants: Standard fine-tuning computes loss over both instruction and completion tokens, whereas SFT computes loss over completion tokens only.
- Tasks and coverage: Fine-tuning covers ARC Easy, SciQ, and OpenBookQA across the full pretrained-checkpoint population, including overtrained models.
- Training configuration: Models are trained for 6 epochs with batch size 8 and learning rate 2 × 10^-5 before test-set evaluation.
- Figures: Figure 6 presents Approach 1 IsoFLOP profiles across scaling budgets for all eight tasks.
- Figures: Figure 7 presents Approach 2 IsoFLOP profiles, while Figure 8 shows the overall checkpoint scaling grid.
E Evaluation Tasks
The evaluation combines four real-world benchmarks with four synthetic tasks spanning language understanding, knowledge, reasoning, and spatial domains. All tasks evaluate models using the negative log-likelihood of the correct answer.
- Evaluation metric: For all tasks, evaluation measures each model’s NLL over the correct answer.
- Real-world benchmarks: The evaluation uses four real-world benchmarks: LAMBADA, ARC Easy, SciQ, and OpenBookQA.
- Real-world benchmarks: OpenBookQA tests multi-step reasoning by combining core science facts with broader common knowledge in four-way multiple-choice questions.
- Synthetic tasks: Four synthetic tasks assess simple knowledge recall, multi-step arithmetic, commonsense causal reasoning, and spatial reasoning.
- Synthetic tasks: Each synthetic task contains 1,000 fill-in-the-blank or short-completion questions generated using GPT-5 and Claude Opus 4.6.
F Fitting T2 Scaling
The paper fits T2 scaling models to empirical checkpoints using two complementary approaches. One directly models loss effects from repeated sampling, while the other combines Chinchilla scaling with pass@k estimation.
- Fitting overview: The fitting procedure models T2 scaling using empirical checkpoint data across sampled values of k.
- Approach 1: Approach 1 fits seven parameters by minimizing squared error between predicted and empirical NLL values using L-BFGS-B with 500 random restarts.
- Approach 2: Approach 2 first fits Chinchilla scaling to NLL values, then fits Beta-regression parameters to empirical pass@k accuracy.