PaperlayerView exact revision on arXiv

Source-linked AI summary

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method

Biao Zhang, Zhongtao Liu, Colin Cherry, Orhan Firat

arXiv:2402.17193v1cs.CLcs.LG

TL;DR

The paper addresses limited understanding of how different LLM finetuning methods scale with model, pretraining-data, parameter, and finetuning-data size. It systematically studies FMT and PET across bilingual translation and multilingual summarization, proposing a multiplicative joint scaling law. The results indicate stronger benefits from LLM model scaling than pretraining-data scaling, ineffective PET parameter scaling, and task- and data-dependent method selection.

  • Problem

    The paper studies the limited understanding of how LLM finetuning scales with pretraining and finetuning factors across different methods.

  • Method

    The study systematically compares FMT, prompt tuning, and LoRA across scaling factors and downstream translation and summarization tasks using a multiplicative joint scaling law.

  • Results

    LLM model scaling benefits finetuning more than pretraining-data scaling, PET parameter scaling is generally ineffective, and optimal finetuning methods depend on task and finetuning-data size.

  • Takeaways & Limitations

    Finetuning-method selection should account for downstream task and available finetuning examples, with PET favored for few examples and FMT for million-scale datasets.

  • Takeaways & Limitations

    The joint scaling law is mainly supported by empirical results on closed-generation tasks without theoretical grounding, and its generalization to other scenarios requires more experimentation.

Abstract

from arXiv · show

While large language models (LLMs) often adopt finetuning to unlock their capabilities for downstream applications, our understanding on the inductive biases (especially the scaling properties) of different finetuning methods is still limited. To fill this gap, we conduct systematic experiments studying whether and how different scaling factors, including LLM model size, pretraining data size, new finetuning parameter size and finetuning data size, affect the finetuning performance. We consider two types of finetuning -- full-model tuning (FMT) and parameter efficient tuning (PET, including prompt tuning and LoRA), and explore their scaling behaviors in the data-limited regime where the LLM model size substantially outweighs the finetuning data size. Based on two sets of pretrained bilingual LLMs from 1B to 16B and experiments on bilingual machine translation and multilingual summarization benchmarks, we find that 1) LLM finetuning follows a powerbased multiplicative joint scaling law between finetuning data size and each other scaling factor; 2) LLM finetuning benefits more from LLM model scaling than pretraining data scaling, and PET parameter scaling is generally ineffective; and 3) the optimal finetuning method is highly task- and finetuning data-dependent. We hope our findings could shed light on understanding, selecting and developing LLM finetuning methods.

1 INTRODUCTION

The paper studies how LLM finetuning performance scales with pretraining and finetuning factors across full-model and parameter-efficient methods. It proposes a multiplicative joint scaling law and finds that model scaling, PET parameter scaling, and method choice behave differently across settings.

  • The study addresses limited understanding of how LLM finetuning scales with model size, pretraining data, finetuning data, and finetuning methods.
  • The experiments compare full-model tuning with parameter-efficient tuning, including prompt tuning and LoRA, while examining finetuning-data scaling and its relationships with other factors.
  • The paper proposes a multiplicative joint scaling law relating finetuning data size to each other scaling factor.
  • Scaling LLM model size benefits finetuning more than scaling pretraining data.
  • Increasing PET parameters does not scale well for LoRA and Prompt, although LoRA shows better training stability.
  • The optimal finetuning method is highly task- and finetuning-data-dependent.

2 SETUP

The setup evaluates FMT, prompt tuning, and LoRA on bilingual translation and multilingual summarization using bilingual LLMs ranging from 1B to 16B parameters. Experiments vary model, pretraining-data, finetuning-data, and PET-parameter scaling factors under specified evaluation procedures.

  • Downstream Tasks: The downstream tasks are WMT14 English-German, WMT19 English-Chinese translation, and multilingual summarization built from German, Spanish, French, and English datasets.
  • LLMs and Pretraining: The pretrained decoder-only bilingual LLMs range from 1B to 16B parameters and use about 280B English/German tokens or 206B English/Chinese tokens.
  • Finetuning Settings: FMT optimizes all LLM parameters, whereas Prompt optimizes a tunable soft prompt prepended to the input.
  • Finetuning Settings: Prompt uses a default soft-prompt length of 100, while LoRA updates pretrained weights through trainable low-rank matrix factors.
  • Finetuning Settings: The study varies four scaling factors; except for LLM model scaling, experiments use the corresponding 1B LLM, and intermediate checkpoints proxy pretraining-data scaling.
  • Evaluation: Evaluation selects the best checkpoint by development-set token-level perplexity and reports test perplexity for scaling-law analyses, with BLEURT and RougeL for generation tasks.

3 WHY MULTIPLICATIVE JOINT SCALING LAW?

The paper compares additive and multiplicative joint scaling formulations after observing power-law finetuning-data scaling across LLM sizes and methods. It adopts the multiplicative formulation because it achieves slightly lower average extrapolation error.

  • Joint scaling analysis: The study treats finetuning data as the pivoting factor and analyzes its relationship with each other scaling factor separately.The factors include LLM model size, pretraining data size, and PET parameter size.
  • Finetuning data scaling: Finetuning performance follows a power law in finetuning data size across model sizes and methods.The fitted formulation uses an irreducible-loss term and is reported to have small predictive errors.
  • Joint-law formulations: The multiplicative and additive joint laws are compared empirically, with α and β representing the impacts of the other factor and finetuning data size.E is a model- and task-dependent irreducible-loss term.
  • Modeling choices: For PET model scaling, newly added parameters are omitted because they represent a tiny and similar proportion across LLM sizes.The paper reports 0.017% added parameters for Prompt and 0.19% for LoRA on a 1B LLM, and finds alternative formulations do not substantially differ.
  • Model selection: The multiplicative joint scaling law generalizes slightly better than the additive formulation on held-out data.Both formulations perform similarly overall, but the multiplicative law has slightly lower average extrapolation error.

4 SCALING RESULTS FOR LLM FINETUNING

Experiments across translation and summarization benchmarks evaluate multiplicative scaling with pretraining data, LLM model size, PET parameter size, and finetuning data. The results favor scaling LLM size over pretraining data, find PET parameter scaling weak, and show method-dependent data scaling.

  • Scaling-law fit: The multiplicative law captures scaling trends across LLM model size, pretraining data size, PET parameter size, and finetuning data.The evaluation uses held-out points and reports fitting and held-out mean absolute deviations.
  • Observed limitations: Extrapolation to 16B is especially mismatched for LoRA and Prompt on WMT19 En-Zh.The paper attributes this to only four model-size data points and pretraining instability in the 16B En-Zh LLM.
  • LLM model and pretraining data scaling: LLM finetuning benefits more from LLM model scaling than pretraining data scaling across tasks and methods.The scaling exponent for model size is often larger than that for pretraining data, although the difference is task-dependent.
  • PET parameter scaling: PET parameter scaling is ineffective for both LoRA and Prompt, with small exponents and inverse scaling in some settings.The paper reports |αt| ≪ 1e−2 and notes training instability when scaling Prompt length; LoRA is more stable and reliable.
  • Finetuning data scaling: Finetuning data has a stronger influence on FMT than PET, while LoRA scales better with data than Prompt.FMT is more data-hungry; LoRA often performs better with more data, whereas Prompt can perform better with only a few thousand examples.
  • PET dependence on pretrained models: PET depends more on LLM model and pretraining data scaling than on finetuning data scaling.The paper attributes this pattern to most LLM parameters remaining frozen during PET and observes that larger LLMs narrow the performance gap between FMT and PET.

5 DISCUSSION

The paper estimates task-dependent critical finetuning data sizes for choosing among methods and examines how finetuning affects zero-shot and few-shot generalization. Method selection depends strongly on downstream task, data availability, and generalization priorities.

  • Method selection: Critical finetuning data sizes are estimated by extrapolating the fitted joint scaling law across LLM model size, pretraining data size, and tasks.The critical point is where two methods achieve equal performance under a specified base-model condition.
  • Method selection: Critical points vary greatly across tasks, so method choices based on one task generalize poorly to others.The paper therefore treats the critical data size as task-specific rather than universal.
  • Method selection: For a few thousand finetuning examples, PET should be considered first; LoRA is preferred with somewhat larger datasets, while FMT suits million-scale datasets.The recommendation reflects the reported stability and data-scalability differences among Prompt, LoRA, and FMT.
  • Generalization: Finetuning can improve generalization to closely related translation tasks, but overall zero-shot quality remains inferior and benefits depend on method and task.The zero-shot score averages translation directions involving French, German, Hindi, Turkish, and Polish into Chinese or German.
  • Generalization: Prompt and LoRA generally preserve more zero-shot generalization than FMT when the base LLM is large.The reported explanation is that PET freezes LLM parameters, allowing learned knowledge to be inherited.

6 RELATED WORK

Related work covers parameter-efficient tuning and neural scaling laws, while positioning this study as complementary to prior work on knowledge transfer versus training from scratch.

  • Parameter-efficient tuning: Parameter-efficient tuning methods reduce the computational burden of updating all parameters in increasingly large LLMs.Prior efforts include adapters, prefix tuning, and prompt tuning, among other efficient tunable modules.
  • Scaling laws: Scaling-law research models neural performance as a power law of model size, data size, or both across domains and architectures.The cited literature spans computer vision, autoregressive generation, neural machine translation, and multilingual translation.
  • Positioning: This study differs from Hernandez et al. by analyzing scaling factors for LLM finetuning rather than comparing finetuning with training from scratch.The paper characterizes the relationship between finetuning data and model, pretraining-data, and PET-parameter sizes.

7 CONCLUSION AND FUTURE WORK

The study systematically evaluates scaling across models, data, parameters, and finetuning methods, finding multiplicative scaling but strong task and data dependence. It also identifies empirical and scope limitations for future work.

  • Conclusion: Experiments span two LLM sets, three downstream tasks, and FMT, Prompt, and LoRA to study multiple finetuning scaling factors.The factors include LLM model size, pretraining data size, finetuning data size, and PET parameter size.
  • Conclusion: A multiplicative joint scaling law describes the relationship between finetuning data size and each other scaling factor.The reported results support this formulation across the study’s evaluated settings.
  • Conclusion: LLM model scaling has greater impact than pretraining-data scaling, while increasing PET parameter size is ineffective.These are the paper’s cross-setting conclusions about the relative scaling behavior of the evaluated factors.
  • Conclusion: Finetuning scaling is highly task- and data-dependent, making the best method for a downstream task less conclusive.The conclusion limits universal method-selection rules across tasks and data regimes.
  • Limitations and future work: The joint scaling law is mainly supported by empirical results on closed-generation tasks without theoretical grounding.Its generalization to other finetuning scenarios remains untested because further experimentation exceeded the computing budget.

A APPENDIX

The appendix documents model configurations, evaluation and optimization details, fitted scaling analyses, and additional generation results. It reports strong PPL–generation-quality correspondence and contrasts FMT with PET on few-shot retention.

  • Configurations: Table 3 defines hyperparameters for different-sized LLMs, including layers, attention heads, head dimension, FFN dimension, and model dimension.The table uses “B” for billion and identifies each architectural dimension explicitly.
  • Additional scaling results: Generation-quality figures report BLEURT for translation and RougeL for summarization while varying model, pretraining-data, PET-parameter, and finetuning-data sizes.The figures cover WMT14 En-De, WMT19 En-Zh, and MLSum in one-shot or five-shot settings.
  • Optimization and evaluation: The optimization continues pretraining from each checkpoint using conditional log-likelihood on target tokens, Adafactor, and a cosine learning-rate schedule.En-De and En-Zh checkpoints were pretrained for 135K and 98K steps, respectively.
  • Critical data sizes: The appendix analyzes critical finetuning data sizes between methods using fitted scaling laws and reports task-specific coefficients without clear cross-setting patterns.Critical sizes correspond to equal performance between two methods under a base-model condition.
  • Scaling-law fit: Pretraining scaling-law extrapolation can fail at 16B for En-Zh, where actual performance is worse than expected, potentially because of pretraining instability.The passage notes that this mismatch may be amplified after finetuning.
  • Evaluation: PPL and BLEURT/RougeL are generally highly correlated, with negative correlations because lower PPL and higher generation scores indicate better quality.The reported table marks statistically significant correlations at p < 0.01.
  • Few-shot capability: FMT often degrades few-shot capability as finetuning data increases, whereas PET retains most few-shot capability across model and pretraining-data sizes.The appendix figures examine one-shot and five-shot performance on translation and summarization benchmarks.
Loading 2402.17193v1…