PaperlayerView exact revision on arXiv

Source-linked AI summary

Late-to-Early Training: LET LLMs Learn Earlier, So Faster and Better

Ji Zhao, Yufei Gu, Shitong Shao, Xun Zhou, Liang Xiang, Zeke Xie

arXiv:2602.05393v1cs.CLcs.LG

TL;DR

Pretraining larger LLMs is highly resource-intensive, motivating whether small pretrained models can guide and accelerate their training. LET uses late-layer representations from a small pretrained model to guide a target model’s early training steps and layers, achieving faster convergence and stronger downstream performance, including up to a 1.6× speedup and nearly 5% higher accuracy in the reported 1.4B-model setting.

  • Problem

    Training larger LLMs is extremely resource-intensive, while whether abundant small pretrained models can generally accelerate their pretraining remains underexplored.

  • Method

    Late-to-Early Training (LET) guides a target model’s early training steps and layers using representations from the late layers of a smaller pretrained LLM.

  • Results

    LET consistently outperforms standard training on most downstream tasks across 1.4B and 7B models, with up to a 1.6× convergence speedup and nearly 5% higher downstream accuracy in the 1.4B setting.

  • Takeaways & Limitations

    Small pretrained models can be used to accelerate larger-model pretraining while improving downstream performance, without imposing architectural constraints.

  • Takeaways & Limitations

    Evaluations primarily cover 1.4B, 3B, and 7B models trained on datasets of up to 20B tokens, so larger-scale validation remains needed.

Abstract

from arXiv · show

As Large Language Models (LLMs) achieve remarkable empirical success through scaling model and data size, pretraining has become increasingly critical yet computationally prohibitive, hindering rapid development. Despite the availability of numerous pretrained LLMs developed at significant computational expense, a fundamental real-world question remains underexplored: \textit{Can we leverage existing small pretrained models to accelerate the training of larger models?} In this paper, we propose a Late-to-Early Training (LET) paradigm that enables LLMs to explicitly learn later knowledge in earlier steps and earlier layers. The core idea is to guide the early layers of an LLM during early training using representations from the late layers of a pretrained (i.e. late training phase) model. We identify two key mechanisms that drive LET's effectiveness: late-to-early-step learning and late-to-early-layer learning. These mechanisms significantly accelerate training convergence while robustly enhancing both language modeling capabilities and downstream task performance, enabling faster training with superior performance. Extensive experiments on 1.4B and 7B parameter models demonstrate LET's efficiency and effectiveness. Notably, when training a 1.4B LLM on the Pile dataset, our method achieves up to 1.6$\times$ speedup with nearly 5\% improvement in downstream task accuracy compared to standard training, even when using a pretrained model with 10$\times$ fewer parameters than the target model.

1 Introduction

LET addresses whether small pretrained open-source models can accelerate the pretraining of much larger LLMs. It uses late-model representations to guide earlier training steps and earlier target-model layers, achieving faster training and stronger downstream performance.

  • 12B-parameter LLM training can require about 72,000 GPU hours, motivating more efficient pretraining paradigms.
  • Existing approaches face overhead or applicability limits, including large-teacher knowledge distillation, modest teacher–student size gaps, preprocessing, and architecture-specific expansion strategies.
  • LET asks whether abundant small pretrained models can generally guide and accelerate larger-model pretraining, including across architectures.
  • LET uses late layers of a smaller pretrained LLM to guide early target-model layers during early training steps through late-to-early-step and late-to-early-layer learning.
  • Up to 1.6× faster improvement in downstream performance was achieved for a 1.4B model on the Pile, even with a model up to 10× smaller than the target.

2 Methodology

LET augments standard language-model pretraining with representation alignment from a small pretrained model. The alignment targets early layers, is emphasized early and decayed over training, and is combined with the target model’s standard objective.

  • LET trains a larger target model M with help from a smaller pretrained model T by aligning early-layer representations of M with late-layer representations of T.
  • LET combines late-to-early-layer learning with late-to-early-step learning, gradually phasing out the pretrained model during training.
  • The target model minimizes standard cross-entropy or negative log-likelihood for predicting each token from its preceding context.
  • The projection loss compares normalized target and pretrained-model hidden states using negative cosine similarity, with a projection applied when their dimensions differ.
  • A linearly decaying weight λ makes representation guidance strong early and zero at the predefined stop step, leaving the primary objective thereafter.

3 Empirical Analysis

Across its empirical studies, LET improves downstream performance and language modeling while accelerating training. Ablations indicate that late-layer-to-early-layer alignment and a moderate alignment weight are central to its robustness.

  • Main results: LET consistently outperforms the baseline across most downstream tasks for both 1.4B and 7B models, with higher average accuracy.The 1.4B setup uses a pretrained model 10× smaller than the target.
  • Main results: LET reduces test perplexity across SmolLM, OPT, and Pythia vocabulary settings, with SmolLM producing the best overall performance.Each comparison uses a consistent vocabulary within its setting.
  • Main results: LET reaches higher performance in fewer than two-thirds of the training steps needed to surpass the baseline, indicating faster convergence.The same acceleration pattern appears for language modeling across three vocabulary settings.
  • Ablation study and analysis: Among six layer-wise alignment strategies, L2E achieves the highest average downstream performance and remains robust during language-modeling training.L2E also yields the lowest final perplexity, while non-L2E strategies show post-alignment perplexity increases.
  • Ablation study and analysis: Alignment strength is best balanced at λ = 0.1: larger values cause performance drops, while λ = 0.01 improves over baseline but remains below λ = 0.1.Higher λ increases cosine similarity, whereas both excessive and insufficient alignment are suboptimal.
  • Additional comparison: LET-1.4B achieves higher performance than Baseline-3B despite using fewer parameters.The reported comparison attributes this efficiency to leveraging representations from the pretrained model during target-model alignment.

4 Discussion

LET is discussed as an efficient alternative to approaches that require architectural expansion, but its scalability remains incompletely validated. The method introduces modest throughput costs while accelerating convergence, and larger-scale evaluation is still needed.

  • LET’s throughput is slightly lower than the baseline, creating a modest computational trade-off.
  • LET’s scalability is not fully established because experiments cover models up to 7B parameters and datasets up to 20B tokens.
  • LET is applied only during early training, so its added computational overhead becomes increasingly negligible as more data is processed.
  • The baseline has 1.078× LET’s early-training throughput, but LET reaches 1.6× faster convergence.

5 Conclusion

The paper presents Late-to-Early Training as a resource-efficient paradigm for strengthening larger LLMs with much smaller pretrained models. Its late-to-early step and layer mechanisms accelerate convergence and improve performance without architectural constraints.

  • LET uses much smaller pretrained models to enhance larger target models, unlike conventional knowledge distillation’s larger-teacher setting.
  • LET combines late-to-early-step learning and late-to-early-layer learning to achieve faster convergence and superior performance without architectural constraints.
  • Experiments on 1.4B-to-7B-parameter models validate LET’s effectiveness and efficiency.

A Experimental Settings and Details

The paper describes its reproducible training, evaluation, and baseline setup, then situates LET among knowledge distillation, weak-to-strong learning, data selection, and model-growth research. LET reverses the usual teacher–student direction and avoids architectural expansion requirements.

  • Experimental settings: Experiments use The Pile with a 2048 batch size, 1280-token sequences, and approximately 2.62 million tokens per step.
  • Evaluation: Models are evaluated on nine downstream tasks and on language-modeling perplexity or loss using The Pile test data.
  • Baselines: Unless otherwise specified, SmolLM-135M is used for 1.4B experiments and SmolLM-1.7B for 7B experiments.
  • Related work: Traditional knowledge distillation generally transfers knowledge from a larger teacher to a smaller student, whereas LET investigates the reverse setting.
  • Related work: Prior learning-dynamics methods achieved success mainly on smaller models and did not propose practical algorithms for LLM training.
  • Related work: LET differs from model-growth and data-selection methods by using a small pretrained model without requiring depth-and-width expansion strategies or complex preprocessing.

C Supplementary Empirical Results

Supplementary experiments test LET across teacher models, stopping thresholds, model scales, and comparisons with RKD and SALT. Across these analyses, LET generally preserves its acceleration and performance advantages, while stopping-threshold choices affect the trade-off between final performance and training time.

  • Additional 1B-scale results: Using OPT-125M or Pythia-160M as teachers still produces substantial improvements and faster convergence for a 1.4B target model despite architectural differences.
  • Downstream comparisons with RKD: RKD consistently underperforms the baseline on both 1.4B and 7B models during downstream-performance evolution.
  • Stopping thresholds: At around 5B tokens, Sstop = 3000 performs better than Sstop = 1500 because alignment remains stronger for longer during early learning.
  • Stopping thresholds: Sstop = 1500 is selected because it yields better final performance while reducing overall training time.
  • Teacher-model generalizability: With Llama-3.2-1B as the teacher for a 7B model, LET still provides significant acceleration and improved final performance, though the gain is smaller than with SmolLM-1.7B.
  • Comparison with SALT: LET achieves superior performance within the same token budget as SALT and exhibits stable training dynamics.

D Time Series Experiments

LET extends beyond language modeling to time series classification, where it significantly outperforms the baseline across diverse datasets. These results support LET's generalizability and effectiveness beyond LLMs.

  • LET significantly outperforms the baseline across diverse time series classification datasets.The evaluation includes datasets spanning gesture, speech, physiological, handwriting, and other time series tasks.
  • The time series baseline fine-tunes Qwen-0.5B on each respective task, while the auxiliary model T is TimesNet.
  • The empirical findings validate LET's generalizability and effectiveness beyond LLMs.

E LM Architecture and Throughput

LET remains effective across substantially heterogeneous language-model architectures and reduces memory overhead during large-batch training. Its representation-focused auxiliary guidance also limits training-time impact because auxiliary models are used only early.

  • Training Efficiency: The training-efficiency comparison reports throughput, wall-clock time, and peak VRAM across methods.Table 4 defines each ratio relative to baseline throughput or resource consumption.
  • LM Architecture: LET improves performance despite substantial heterogeneity in hidden sizes, layer counts, attention heads, activation functions, and attention mechanisms.The evaluated activation functions include ReLU, GeLU, SiLU, and SwiGLU; attention variants include Full and GQA.
  • Training Efficiency: LET achieves lower peak VRAM than auxiliary-model methods when training with large batch sizes.The reduction stems from aligning representations rather than the larger logit space.
  • Training Efficiency: LET and SALT require auxiliary models only during early training, minimally affecting wall-clock time and throughput relative to baseline.LET has slightly higher wall-clock time than SALT, while its lower peak VRAM under large-batch training supports scaling potential.
  • Alignment Objectives: LET evaluates alternative alignment objectives, including logsum loss, because the models have substantial capacity differences.The comparison also considers CCA alignment and tokenizer mismatch in the reported downstream-performance table.
  • Alignment Objectives: Logsum loss consistently outperforms Baseline, RKD, SALT, and LET in average downstream task performance.It emphasizes regions where the target and auxiliary representations diverge most, providing explicit alignment guidance.

H Theoretical Analysis

The theoretical analysis uses a simplified deep linear network and Hessian-based curvature bounds to compare alignment depths. Under a uniform Hessian-block assumption, earlier alignment has a smaller additional-curvature bound and therefore a smoother theoretical landscape.

  • Analysis Setup: LET's theoretical analysis compares optimization landscapes for early-layer and non-early-layer alignment.The analysis focuses on a simplified deep linear network with equal representation dimension for models M and T.
  • Analysis Setup: The total objective combines the language-modeling loss LNLL with the projection loss Lproj weighted by λ.The curvature analysis isolates Lproj because LNLL is shared across alignment settings.
  • Hessian Structure: The alignment loss at layer k depends on parameters in layers 0 through k−1, producing a structured Hessian block form.Blocks associated with later layers are zero because the representation at layer k does not depend on those parameters.
  • Curvature Bound: Under a uniform Frobenius-norm bound C for Hessian blocks, the projection-loss curvature satisfies ∥Hproj(k)∥F ≤ kC.The Frobenius norm is used as a measurable proxy for curvature magnitude and bounds the spectral norm.
  • Curvature Bound: For k1 < k2 < L, earlier alignment layers have smaller theoretical curvature upper bounds than later alignment layers.This follows from the bound's linear dependence on alignment depth k.
  • Conclusion: The analysis concludes that LET preserves more of the original optimization landscape and yields smoother optimization, which is also observed empirically in modern architectures.The theoretical result relies on the simplified deep linear setting and uniform Hessian-block bound assumption.

I Failure Mode Analysis and Layer Selection Strategies

LET's effectiveness depends on the quality of the small model and the selected alignment layers. GPT-2 variants underperform baseline, while aligning the auxiliary model's final layer with the target's third layer gives the best reported layer-pair performance.

  • Scope: The failure-mode section examines both LET's limitations and the effect of layer-selection strategies on final performance.
  • Failure Modes: LET underperforms baseline when GPT-2 is used as the small model T.LET-GPT2-Medium performs better than LET-GPT2-Small and RKD, but all variants remain below baseline.
  • Failure Modes: GPT-2's representations may provide ineffective alignment signals because its training data has a late-2017 cutoff and potentially lower quality.Despite this degradation, LET consistently outperforms RKD, indicating greater robustness to model quality.
  • Layer Selection: Aligning the final layer of T with the target model's third layer, L1-F3, achieves the best performance among evaluated layer pairings.L1-F5 performs worse than L1-F3, while first-layer alignment may encode more input-specific information.

J Descriptions of Evaluation Tasks

The evaluation suite covers language understanding, commonsense reasoning, science knowledge, and question answering across varied task formats. Several benchmarks require combining factual knowledge with inference or contextual understanding.

  • Language understanding: LAMBADA tests broad contextual understanding by requiring prediction of the final word in a passage.The passages are filtered so the answer depends on context beyond the final sentence.
  • Commonsense reasoning: PIQA, HellaSwag, and Winogrande evaluate different forms of commonsense reasoning, including physical plausibility, sentence completion, and pronoun resolution.HellaSwag uses counter-intuitive distractors, while Winogrande requires commonsense knowledge for coreference resolution.
  • Science and question answering: SciQ, OpenbookQA, and ARC assess science question answering through factual recall, commonsense reasoning, and inference.ARC separates easier questions from challenge questions requiring more complex inference or broader background knowledge.
  • Question answering: BoolQ evaluates binary question answering by asking whether an answer is entailed by a short passage.The task uses yes/no decisions over passage-based questions.
Loading 2602.05393v1…