PaperlayerView exact revision on arXiv

Source-linked AI summary

SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Tuo Zhao

arXiv:1911.03437v5cs.CLcs.LGmath.OC

TL;DR

Limited downstream labels and large pretrained-model capacity make fine-tuning vulnerable to overfitting and loss of pretrained knowledge. SMART combines smoothness-inducing adversarial regularization with Bregman proximal optimization, and the authors report state-of-the-art performance across multiple NLP benchmarks. The paper also reports applicability to domain adaptation.

  • Problem

    Limited downstream data and high-capacity pretrained models make aggressive fine-tuning prone to overfitting and forgetting pretrained knowledge.

  • Method

    SMART combines smoothness-inducing adversarial regularization to control model capacity with Bregman proximal point optimization to constrain updates.

  • Results

    SMART improves performance across GLUE, SNLI, SciTail, and ANLI with pretrained models including BERT, MT-DNN, and RoBERTa.

  • Takeaways & Limitations

    The framework is reported to alleviate overfitting and aggressive updating and to apply to domain adaptation and other transfer-learning problems.

  • Takeaways & Limitations

    Due to computational-resource limitations, the experiments trained jointly on only MNLI, RTE, QNLI, SST, and MRPC, rather than all GLUE tasks.

Abstract

from arXiv · show

Transfer learning has fundamentally changed the landscape of natural language processing (NLP) research. Many existing state-of-the-art models are first pre-trained on a large text corpus and then fine-tuned on downstream tasks. However, due to limited data resources from downstream tasks and the extremely large capacity of pre-trained models, aggressive fine-tuning often causes the adapted model to overfit the data of downstream tasks and forget the knowledge of the pre-trained model. To address the above issue in a more principled manner, we propose a new computational framework for robust and efficient fine-tuning for pre-trained language models. Specifically, our proposed framework contains two important ingredients: 1. Smoothness-inducing regularization, which effectively manages the capacity of the model; 2. Bregman proximal point optimization, which is a class of trust-region methods and can prevent knowledge forgetting. Our experiments demonstrate that our proposed method achieves the state-of-the-art performance on multiple NLP benchmarks.

1 Introduction

NLP transfer learning addresses scarce labeled data by adapting high-capacity pretrained language models, but aggressive fine-tuning can overfit and forget pretrained knowledge. SMART combines smoothness-inducing regularization with Bregman proximal optimization and reports strong performance across NLP benchmarks.

  • Motivation: Transfer learning uses high-resource out-of-domain data to support low-resource target tasks through pretraining followed by fine-tuning.The pretrained model is adapted to the target domain during the second stage.
  • Motivation: Limited target data and high model complexity make aggressive fine-tuning prone to overfitting and poor generalization.Existing remedies often rely on heuristic learning-rate schedules, layer freezing, or added layers.
  • Proposed framework: SMART controls model capacity with smoothness-inducing adversarial regularization and limits aggressive updates with Bregman proximal point optimization.The regularizer encourages locally stable outputs, while the optimizer constrains updates near the previous iterate.
  • Results: 356M-parameter SMART achieves three state-of-the-art GLUE results without an ensemble and compares favorably with models including 11B-parameter T5.The authors also report improved training stability and generalization across multiple NLP tasks.
  • Contributions: SMART introduces smoothness-inducing adversarial regularization and proximal point optimization into large-scale language-model fine-tuning, achieving state-of-the-art results on several NLP benchmarks.The contribution covers benchmarks including GLUE, SNLI, and SciTail.

2 Background

The background reviews pretrained transformer language models, standard downstream fine-tuning, and heuristic defenses against overfitting. It positions SMART relative to adversarial fine-tuning and proximal-point optimization.

  • Pretrained models: Transformer-based language models such as BERT are pretrained without human annotation and then adapted to downstream tasks.Pretraining uses large corpora and unsupervised learning tasks.
  • Fine-tuning: Existing fine-tuning heuristics include small or triangular learning rates, few iterations, selective layer adaptation, and added task-specific layers.These methods aim to reduce overfitting during downstream adaptation.
  • Related methods: SMART’s adversarial regularization is related to methods developed for semi-supervised learning, domain adaptation, and image classification, but targets language-model fine-tuning.The shared regularization idea is used for different applications and motivations.
  • Related methods: SMART’s optimization belongs to a broader family of Bregman proximal point methods, including vanilla, generalized, and accelerated variants.These methods provide the optimization framework underlying SMART’s update strategy.
  • Related methods: FreeLB adapts robust adversarial training, whereas SMART focuses on local smoothness and reports significant performance improvement.The paper provides further comparison in its experiments.

3 The Proposed Method

SMART fine-tunes pretrained language models by combining local smoothness regularization with Bregman proximal updates. The two components respectively control model complexity and constrain successive parameter updates.

  • Framework: SMART is a fine-tuning framework built from Smoothness-inducing Adversarial Regularization and Bregman Proximal Point Optimization.The name SMART abbreviates the two ingredients.
  • Smoothness-Inducing Adversarial Regularization: The objective combines task loss with an explicit smoothness regularizer to control model complexity during fine-tuning.The regularizer is weighted by λs and is computed from perturbations around target-task inputs.
  • Smoothness-Inducing Adversarial Regularization: For classification, the smoothness loss uses symmetrized KL-divergence; for regression, it uses squared loss, with the adversarial maximization solved by projected gradient ascent.The perturbation radius is controlled by ϵ.
  • Novelty: SMART applies a smoothness-inducing regularizer to pretrained-language-model fine-tuning, which the authors identify as a novel application of this regularizer.Earlier uses targeted semi-supervised learning, domain adaptation, and image classification.
  • Smoothness-Inducing Adversarial Regularization: The smoothness regularizer encourages outputs to remain stable under small input perturbations, promoting local smoothness around training examples.The paper relates this behavior to local Lipschitz continuity and improved low-resource generalization.
  • Bregman Proximal Point Optimization: Bregman proximal updates penalize deviation from the previous iterate, creating trust-region-type steps that can retain pretrained knowledge.A larger μ strengthens this constraint, and each subproblem can be approximated without full convergence.
  • Bregman Proximal Point Optimization: Momentum Bregman proximal point optimization accelerates updates using an exponential moving average of parameters.The momentum parameter β lies in (0,1), and the method is also called Mean Teacher.

4 Experiment – Main Results

SMART is evaluated on GLUE using BERTBASE and RoBERTaLARGE implementations against pretrained-model, adversarial-training, and leaderboard baselines. It consistently improves development and test performance, including state-of-the-art results with a single 356M-parameter model.

  • Experimental setup: SMART is evaluated with BERTBASE and RoBERTaLARGE against pretrained-model, adversarial-training, and leaderboard baselines.The experiments use SMARTBERT and SMARTRoBERTa, with comparisons including BERT, RoBERTa, PGD, FreeAT, FreeLB, and other submitted GLUE models.
  • Experimental setup: Table 1 reports GLUE development-set results, with tasks grouped by pretrained model and the best single-model result highlighted for each task.The table distinguishes BERTBASE and RoBERTaLARGE groups; missing results are marked with “-”.
  • Main comparisons: SMART consistently outperforms BERT and RoBERTa across all 8 GLUE tasks, including 85.6% versus 84.5% accuracy for MNLI in-domain.SMARTBERT also reaches 86.0% versus 84.4% on MNLI out-domain, while SMARTRoBERTa reaches 91.1% versus 90.2% on MNLI in-domain.
  • Main comparisons: SMART outperforms FreeLB on 6 of 8 GLUE tasks: MNLI, RTE, QNLI, MRPC, SST-2, and STS-B.The comparison covers adversarial-training baselines and identifies the six tasks where SMART exceeds FreeLB.
  • Leaderboard results: A single 356M-parameter SMART model sets state-of-the-art results on SST-2, MRPC, and STS-B, while MT-DNN-SMART reaches 89.9% on GLUE.The comparison includes T5 with 11 billion parameters and ensemble models; the reported MT-DNN-SMART result combines SMART with multi-task learning.

5 Experiment – Analysis and Extension

The experiments analyze SMART’s components, compatibility with multi-task learning, domain adaptation, and robustness to adversarial NLI data. Across these settings, SMART improves generalization and benchmark performance, especially under ambiguity and limited data.

  • 5.1 Ablation Study: Removing either smooth regularization or Bregman proximal optimization lowers MNLI performance, showing that the two components complement each other.Removing smooth regularization changes MNLI from 85.6% to 84.8%, while removing the proximal optimization changes it to 85.4%.
  • 5.1 Ablation Study: SMART’s generalization benefits are larger on the small RTE and MRPC datasets, consistent with its role in preventing overfitting.The ablation analysis specifically identifies RTE and MRPC as small datasets where SMART provides greater generalization benefits.
  • 5.2 Error Analysis: SMARTRoBERTa outperforms RoBERTa across agreement categories, with larger gains on highly ambiguous samples.The analysis groups MNLI development examples by annotation agreement and evaluates both accuracy and KL-divergence.
  • 5.3 SMART with Multi-task Learning: SMART and MT-DNN both outperform BERT across five GLUE tasks, while their combination improves selected results and strengthens multi-task generalization.MT-DNN-SMART reaches 85.7% on MNLI and 80.2% on RTE, improving over the best separate MT-DNN and SMART results by 0.1% and 1.1%, respectively.
  • 5.5 Results on SNLI and SciTail: In domain adaptation, MT-DNN-SMART outperforms the BERT baseline and achieves improvements over MT-DNN on SNLI and SciTail.The combined model is reported to outperform current state-of-the-art performance on the SNLI/SciTail test sets.
  • 5.6 Robustness: On ANLI, SMARTRoBERTa-LARGE achieves the best combined-data performance and exceeds the RoBERTa-LARGE baseline by 3.4% on development data and 7.4% on test data.In the ANLI-only setting, it also improves over RoBERTa-LARGE by 5.2%: 57.1% versus 51.9%.

6 Conclusion

SMART is a robust and efficient framework for fine-tuning large pre-trained language models that combines smoothness-inducing adversarial regularization with Bregman proximal point optimization. The framework alleviates overfitting and aggressive updating, improves performance across multiple NLP benchmarks, and also supports domain adaptation.

  • SMART combines smoothness-inducing adversarial regularization with Bregman proximal point optimization for principled fine-tuning.These are the framework’s two stated ingredients.
  • SMART alleviates overfitting and aggressive updating during fine-tuning.
  • SMART improves performance on GLUE, SNLI, SciTail, and ANLI with state-of-the-art pre-trained models.The cited models include BERT, MT-DNN, and RoBERTa.
  • The framework also produces significant performance improvements for domain adaptation and can generalize to other transfer learning problems.The latter is stated as a supported generalization of the fine-tuning framework.

7 Datasets

This section summarizes four benchmarks—GLUE, SNLI, SciTail, and ANLI—used to evaluate the framework. SciTail is used specifically for domain adaptation and is characterized by challenging, lexically similar premise–hypothesis pairs.

  • The dataset section summarizes the GLUE, SNLI, SciTail, and ANLI benchmarks.
  • SciTail: SciTail examples pair science-question hypotheses and answer-candidate premises with relevant web sentences retrieved from a large corpus.
  • SciTail: High lexical similarity between SciTail premises and hypotheses makes the dataset particularly difficult.
  • SciTail: SciTail is used only for domain adaptation in this study.
  • ANLI: ANLI is a large-scale natural language inference benchmark collected through an iterative adversarial human-and-model-in-the-loop procedure.

8 Hyperparameters

SMART is generally insensitive to moderate hyper-parameter choices, but excessively strong or weak regularization is unsuitable. The p = ∞ perturbation norm reduces tuning effort, while larger iteration settings yield only minor gains at substantially higher computational cost.

  • Performance changes only slightly when λs, µ, and ϵ remain within [1,10], [1,10], and [10^-5,10^-4], respectively.
  • Regularization is unreasonably strong when λs ≥100, µ ≥100, or ϵ ≥10^-3, and unreasonably weak below λs ≤0.1, µ ≤0.1, or ϵ ≤10^-6.
  • The algorithm is insensitive to σ, with any σ ≤ϵ working well.
  • Using p = ∞ makes perturbation size independent of dimensionality and requires less tuning than p = 2.Other p values use computationally inefficient projections.
  • Larger S or Tex provides only minor improvement while increasing computation; with S = Tex = 1, training takes about 3 times longer and approximately doubles GPU memory usage.This setting adds 3 forward and 3 backward passes per iteration relative to direct fine-tuning.
  • β is set to 0.99 for the first 10% of updates and 0.999 thereafter, which works well in practice.
Loading 1911.03437v5…