PaperlayerView exact revision on arXiv

Source-linked AI summary

Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning

Runxin Xu, Fuli Luo, Zhiyuan Zhang, Chuanqi Tan, Baobao Chang, Songfang Huang, Fei Huang

arXiv:2109.05687v1cs.CLcs.AI

TL;DR

Fine-tuning large pretrained language models with limited data is difficult because aggressive updates can overfit and weaken transfer. Child-Tuning masks non-child gradients so only a selected parameter subset is updated, and experiments report stronger benchmark and transfer performance than vanilla and prior fine-tuning methods.

  • Problem

    Adapting large pretrained language models with limited labeled data remains challenging because aggressive fine-tuning can overfit and generalize poorly to out-of-domain data or related tasks.

  • Method

    Child-Tuning updates a selected child network by masking non-child gradients during backpropagation, with task-free and task-driven ways to detect it.

  • Results

    Child-Tuning outperforms vanilla fine-tuning across four pretrained models and reports up to 8.60 average score improvement on CoLA/RTE/MRPC/STS-B.

  • Takeaways & Limitations

    Child-Tuning can improve downstream adaptation and generalization across in-domain, out-of-domain, and cross-task transfer scenarios.

  • Takeaways & Limitations

    The task-driven variant assumes that parameters with higher Fisher information are more important for the target task.

Abstract

from arXiv · show

Recent pretrained language models extend from millions to billions of parameters. Thus the need to fine-tune an extremely large pretrained model with a limited training corpus arises in various downstream tasks. In this paper, we propose a straightforward yet effective fine-tuning technique, Child-Tuning, which updates a subset of parameters (called child network) of large pretrained models via strategically masking out the gradients of the non-child network during the backward process. Experiments on various downstream tasks in GLUE benchmark show that Child-Tuning consistently outperforms the vanilla fine-tuning by 1.5~8.6 average score among four different pretrained models, and surpasses the prior fine-tuning techniques by 0.6~1.3 points. Furthermore, empirical results on domain transfer and task transfer show that Child-Tuning can obtain better generalization performance by large margins.

1 Introduction

Large pretrained language models remain difficult to adapt with limited data because aggressive fine-tuning can overfit and generalize poorly. Child-Tuning addresses this by updating only a strategically selected child network while retaining whole-model forward computation, with task-free and task-driven variants.

  • Motivation: Limited labeled data makes adapting large pretrained language models challenging, as conventional fine-tuning can overfit and generalize poorly to new domains or related tasks.The paper links these issues to the models’ large capacity and aggressive fine-tuning.
  • Approach: Child-Tuning updates only a subset of model parameters, called the child network, by masking non-child gradients during backpropagation.The full network is still used during the forward pass.
  • Approach: CHILD-TUNINGF detects the child network without task data, using a Bernoulli distribution that introduces gradient noise and regularization.The stated goal is to reduce overfitting on small datasets and improve generalization.
  • Approach: CHILD-TUNINGD uses downstream data to identify task-related parameters and freezes non-child parameters at their pretrained values.It applies a task-specific mask to reduce the model’s hypothesis space while adapting to downstream tasks.
  • Results: Up to 8.60 average score improvement is reported on CoLA/RTE/MRPC/STS-B over vanilla fine-tuning across different pretrained language models.The paper also reports better out-of-domain and cross-task transfer, stronger results than prior fine-tuning methods, and further gains when methods are integrated.
  • Results: Child-Tuning is presented as orthogonal to prior fine-tuning methods, allowing combinations that can further improve fine-tuning performance.This compatibility is reported as an additional contribution beyond the standalone variants.

2 Methodology

CHILD-TUNING updates only a selected subset of parameters while still forwarding through the full pretrained model. It offers task-free and task-driven ways to choose this child network, with theoretical justification for improved optimization and generalization.

  • Overview of CHILD-TUNING: CHILD-TUNING inserts a gradient mask during backpropagation so only the child network is updated while the full model remains active in the forward pass.This differs from pruning because pretrained knowledge in the entire network is still used.
  • Overview of CHILD-TUNING: Vanilla fine-tuning computes loss gradients and applies gradient descent to all model parameters.CHILD-TUNING changes this update by masking gradients before optimization.
  • Task-Free Variant: CHILD-TUNINGF: CHILD-TUNINGF samples a task-free gradient mask from a Bernoulli distribution with reservation probability pF.Larger pF reserves a larger child network, while pF = 1 recovers vanilla fine-tuning and reserved gradients are scaled by 1/pF.
  • Task-Free Variant: CHILD-TUNINGF: Theorem 1 states that gradient variance strictly decreases as pF increases, while Theorem 2 links higher variance to convergence at flatter local minima.The analysis further states that CHILD-TUNINGF decreases the generalization error bound.
  • Task-Driven Variant: CHILD-TUNINGD: CHILD-TUNINGD selects task-relevant parameters using diagonal empirical Fisher information estimated from downstream training data.The child network contains parameters with the highest estimated information, and the selected network is kept fixed during fine-tuning.
  • Task-Driven Variant: CHILD-TUNINGD: Keeping the non-child network at its pretrained weights reduces the hypothesis space, alleviates overfitting, and substantially maintains generalization ability.This describes the stated motivation for fixing the task-driven child network during fine-tuning.

3 Experiments

Experiments evaluate CHILD-TUNING across GLUE tasks, pretrained models, low-resource settings, and transfer scenarios. The method consistently improves fine-tuning and preserves stronger domain- and task-generalization performance than vanilla fine-tuning.

  • Experimental setup: Experiments cover GLUE classification and similarity tasks, additional NLI datasets, four large pretrained models, and results averaged over 10 random seeds.The models include BERTLARGE, XLNetLARGE, RoBERTaLARGE, and ELECTRALARGE.
  • GLUE benchmark: 8.60 average score improvement is achieved on ELECTRA over vanilla fine-tuning, while CHILD-TUNINGD reaches up to 1.50 on BERT.CHILD-TUNING improves across tasks and pretrained models, with the task-driven variant performing best.
  • Domain generalization: CHILD-TUNING outperforms vanilla fine-tuning across out-of-domain NLI datasets, with CHILD-TUNINGD improving up to 1.53/0.81 average score for models trained on MNLI/SNLI.CHILD-TUNINGF improves 1.11/0.35 average score under the same training conditions.
  • Task generalization: 7.06 average score improvement is obtained when transferring from MRPC to four GLUE tasks with CHILD-TUNINGD, compared with 4.58 for CHILD-TUNINGF.The reported vanilla-fine-tuning baseline is 58.95, increasing to 63.53 and 66.01.

4 Analysis and Discussion

Analysis compares CHILD-TUNING with prior fine-tuning methods, examines low-resource behavior, distinguishes it from pruning, and studies task-driven child-network structure. Results favor task-driven parameter selection and show that non-child parameters remain useful during inference.

  • Comparison with prior methods: CHILD-TUNINGD improves the total average score by 1.50 over prior fine-tuning methods, while combining it with R3F yields a 1.84 improvement.CHILD-TUNINGF performs on par with Mixout and R3F in the reported comparison.
  • Low-resource scenarios: With 1k training examples, CHILD-TUNINGF improves average score by 1.42 and CHILD-TUNINGD by up to 2.24 over vanilla fine-tuning.The analysis attributes the larger task-driven gain to decreasing the model’s hypothesis space.
  • Difference from model pruning: Pruning parameters outside the child network decreases the average score by 33.89 points and causes model collapse on CoLA.CHILD-TUNING instead retains non-child parameters during training and inference, preserving pretrained knowledge.
  • Child-network selection: Selecting a random child network improves average score by 0.18, whereas selecting the lowest-Fisher-information parameters decreases it by 6.65 relative to highest-Fisher selection.These ablations support restricting updates while favoring task-relevant parameters.
  • Task-driven network structure: Similar tasks generally have higher overlapping ratios between their task-driven child networks, while different task formats show lower overlap.NLI task pairs such as RTE–QNLI and QNLI–MNLI show notably higher overlap; CoLA–MRPC shows relatively low overlap.

5 Related Work

Related work spans large pretrained language models, effective fine-tuning methods, and parameter-efficient adaptation. CHILD-TUNING differs by targeting generalizable fine-tuning through selective updates within the pretrained model without adding modules.

  • Pretrained language models: Large pretrained language models have expanded in scale and now support a broad range of NLP tasks through pretraining and fine-tuning.The related work lists BERT, GPT-2, GPT-3, ELECTRA, XLNet, RoBERTa, and BART.
  • Effective and generalizable fine-tuning: Prior effective fine-tuning methods regularize deviation from pretrained weights or otherwise constrain adaptation to address overfitting and poor generalization.Examples include Weight Decay, Mixout, RecAdam, and R3F.
  • Parameter-efficient fine-tuning: Parameter-efficient methods aim to fine-tune as few parameters as possible, whereas CHILD-TUNING targets effective and generalizable fine-tuning.Adapters add trainable modules, while Diff-Pruning sparsifies a difference vector; CHILD-TUNING detects an internal child network without extra parameters.

6 Conclusion

The conclusion presents CHILD-TUNING as a selective-gradient fine-tuning method for large pretrained language models. Across downstream tasks and transfer settings, it improves performance and generalization, and can be combined with other fine-tuning techniques.

  • Conclusion: CHILD-TUNING masks non-child gradients so that only a subset of parameters is updated during fine-tuning.CHILD-TUNINGF and CHILD-TUNINGD identify the child network in task-free and task-driven ways, respectively.
  • Conclusion: Experiments across four pretrained language models show gains over vanilla fine-tuning and prior methods while enhancing domain and task generalization.The conclusion also reports that CHILD-TUNING is orthogonal to most prior techniques and can be integrated with them.

A GLUE Benchmark Introduction

The study evaluates Child-Tuning on eight GLUE datasets spanning single-sentence, inference, similarity, and paraphrase tasks, excluding WNLI because of construction issues.

  • 8 GLUE datasets cover single-sentence, inference, similarity, and paraphrase tasks.
  • WNLI is excluded because of issues with its dataset construction, following common practice in prior studies.
  • Metrics are reported separately for each dataset.

B Settings for Different Pretrained Language Models

Experiments use several large pretrained Transformer language models, with model-specific hyperparameters and implementation details reported for the GLUE evaluation.

  • Four pretrained models are evaluated: BERTLARGE, XLNetLARGE, RoBERTaLARGE, and ELECTRALARGE.
  • Training settings are listed in Table 7, while optimization uses AdamW with gradient clipping and maximum sequence length 128.
  • All evaluated pretrained models are Transformer-based.
  • XLNet uses token permutations and autoregressive generation while capturing bidirectional context.
  • RoBERTa is an optimized BERT variant using dynamic masking, larger batches, longer training, and no next-sentence prediction.
  • ELECTRA uses a generator and discriminator, training the discriminator to distinguish generated tokens from original tokens.

C Settings for Other Fine-tuning Methods

The paper compares Child-Tuning with regularization and parameter-update baselines, using stated loss formulations, replacement strategies, and grid-searched hyperparameters.

  • The comparison includes Weight Decay, Top-K Fine-tuning, Mixout, RecAdam, and R3F.
  • Weight Decay penalizes the L2 distance between fine-tuned and pretrained parameters.
  • Top-K Fine-tuning updates only the top K layers and classification layer while freezing the remaining bottom layers.
  • Mixout randomly replaces parameters with pretrained weights during fine-tuning according to probability p.
  • RecAdam changes the relative weights of two loss terms during training and is treated as an advanced form of Weight Decay.
  • R3F adds noise to input embeddings and minimizes symmetrical KL divergence between outputs for original and noisy inputs.

D Label Mapping in Task Generalization

The task-generalization setup maps labels across related natural-language-inference datasets, while the theory links gradient variance, flat minima, and generalization bounds for CHILD-TUNINGF.

  • Label Mapping in Task Generalization: SciTail maps source neutral and contradiction labels to target neutral, while MNLI and SNLI retain entailment, neutral, and contradiction.
  • Label Mapping in Task Generalization: QQP’s duplicate and not-duplicate labels are interpreted as entailment and neutral, respectively.
  • Theoretical Details: Theorem 1 states that CHILD-TUNINGF’s gradient variance strictly decreases as the gradient-reserving probability p increases.
  • Theoretical Details: The paper concludes that CHILD-TUNINGF decreases the generalization error bound through increased variance and flatter minima.
  • Theoretical Details: The analysis assumes Gaussian gradient or update distributions and specifies pretrained parameters, local minima, Hessian sharpness, and training-set probability conditions.
  • Theoretical Details: Theorem 2 relates higher update variance to convergence at flatter local minima characterized by smaller Hessian maximum eigenvalue ρ.

E.1 Proof of Theorem 1

The proof analyzes gradient-induced updates near a local minimum and derives conditions under which the algorithm remains there, together with a generalization error bound.

  • The quantity g√|B| is a strictly decreasing function of p.
  • A Taylor expansion of the training loss around a local minimum yields the escape analysis after omitting higher-order terms.The proof uses ∇_wL(w) = 0_k at the local minimum.
  • If the escape probability satisfies the stated bound, the algorithm does not escape the local minimum and can converge to it.
  • The proof then invokes a prior result to establish the following generalization error bound with probability 1−δ over the training set.The cited lemma uses a Gaussian prior over parameters and k denotes the number of parameters.
Loading 2109.05687v1…