PaperlayerView exact revision on arXiv

Source-linked AI summary

FreeLB: Enhanced Adversarial Training for Natural Language Understanding

Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, Jingjing Liu

arXiv:1909.11764v5cs.CLcs.LG

TL;DR

Language models need stronger generalization and robustness, while constructing label-preserving text perturbations is difficult. FreeLB addresses this by perturbing word embeddings and minimizing adversarial risk across nearby regions during finetuning. It improves Transformer-based models across GLUE, ARC, and CommonsenseQA, while adversarial training still incurs significant overhead compared with vanilla SGD.

  • Problem

    Language-model adversarial training seeks label-preserving perturbations that improve generalization and robustness, but constructing actual adversarial language examples remains difficult.

  • Method

    FreeLB adds gradient-based perturbations to continuous word embeddings and efficiently minimizes the resulting adversarial risk around input samples.

  • Results

    FreeLB improves Transformer-based models across several datasets, including GLUE, ARC, and CommonsenseQA, with GLUE scores rising from 78.3 to 79.4 for BERT-base and from 88.5 to 88.8 for RoBERTa-large.

  • Takeaways & Limitations

    FreeLB increases embedding-space robustness and generalization ability for the evaluated Transformer-based language-understanding and reasoning tasks.

  • Takeaways & Limitations

    Adversarial training still takes significant overhead compared with vanilla SGD, and dropout makes successive perturbation updates optimize different effective networks.

Abstract

from arXiv · show

Adversarial training, which minimizes the maximal risk for label-preserving input perturbations, has proved to be effective for improving the generalization of language models. In this work, we propose a novel adversarial training algorithm, FreeLB, that promotes higher invariance in the embedding space, by adding adversarial perturbations to word embeddings and minimizing the resultant adversarial risk inside different regions around input samples. To validate the effectiveness of the proposed approach, we apply it to Transformer-based models for natural language understanding and commonsense reasoning tasks. Experiments on the GLUE benchmark show that when applied only to the finetuning stage, it is able to improve the overall test scores of BERT-base model from 78.3 to 79.4, and RoBERTa-large model from 88.5 to 88.8. In addition, the proposed approach achieves state-of-the-art single-model test accuracies of 85.44\% and 67.75\% on ARC-Easy and ARC-Challenge. Experiments on CommonsenseQA benchmark further demonstrate that FreeLB can be generalized and boost the performance of RoBERTa-large model on other tasks as well. Code is available at \url{https://github.com/zhuchen03/FreeLB .

1 INTRODUCTION

FreeLB reframes adversarial training as a way to improve language-model generalization, using embedding perturbations and diversified adversarial samples. Experiments report gains across GLUE and other language-understanding benchmarks.

  • Adversarial training can improve both generalization and robustness in language models, unlike reported effects in computer vision.The introduction contrasts language-model results with accuracy drops of up to 10% on CIFAR-10 and 15% on ImageNet.
  • FreeLB adds adversarial perturbations to word embeddings and minimizes the resulting adversarial loss around input samples.It uses free training strategies to create diversified adversarial samples under different norm constraints.
  • FreeLB is reported to outperform standard PGD, FreeAT, and YOPO across the evaluated datasets and models.The experiments cover state-of-the-art language-understanding models and tasks.
  • GLUE overall scores rise from 78.3 to 79.4 for BERT-base and from 88.5 to 88.8 for RoBERTa-large with FreeLB.The method achieves the best results on most RoBERTa-large GLUE subtasks.

2 RELATED WORK

Related work establishes adversarial training as a robustness technique while highlighting its computational cost and the difficulty of constructing meaningful text perturbations. Prior NLP studies explored adversarial examples through semi-supervised training, attacks, and model debugging.

  • PGD-based adversarial training solves a minimax cross-entropy problem through projected gradient ascent followed by stochastic-gradient descent.It is widely regarded as effective because it largely avoids obfuscated gradients.
  • PGD-based training can require substantially higher cost than conventional training because small-step loss surfaces may remain highly convoluted and nonlinear.The “free” algorithm updates model parameters and perturbations in one backward pass to mitigate this cost.
  • Prior text-adversary work used black-box heuristics, distracting sentences, latent-space GANs, and word-level manipulations to induce misclassification.These approaches focused on constructing adversarial examples rather than the embedding-space training procedure introduced here.
  • Earlier NLP studies applied adversarial methods to semi-supervised classification, robustness training, debugging, and related attack settings.These studies establish that adversarial examples can support both evaluation and model improvement in language tasks.

3 ADVERSARIAL TRAINING FOR LANGUAGE UNDERSTANDING

The paper develops FreeLB to improve language-model generalization by perturbing word or subword embeddings and optimizing across multiple adversarial regions, while reducing adversarial-training overhead.

  • Embedding-space adversarial training: FreeLB creates virtual adversarial examples by perturbing concatenated word or subword embeddings while leaving segment and position embeddings unchanged.The perturbation is constrained to preserve semantics and assumes the model prediction should remain unchanged.
  • PGD for adversarial training: Standard PGD adversarial training minimizes worst-case loss within a norm ball using projected gradient ascent for the inner maximization and SGD for the outer minimization.For neural networks, the inner problem is non-concave and the outer problem is non-convex, but this procedure can nevertheless be solved reliably in practice.
  • PGD for adversarial training: K-step PGD requires K forward-backward passes, increasing adversarial-training runtime by an order of magnitude over conventional training.The added cost is especially substantial when training large language models.
  • Free large-batch adversarial training: FreeLB performs multiple PGD iterations while accumulating parameter gradients, then updates the model parameters once using the accumulated gradients.This procedure approximately optimizes loss across adversarial embeddings from several ascent steps.
  • Free large-batch adversarial training: FreeLB is equivalent to replacing each original batch with a K-times larger virtual batch containing embeddings X + δ0 through X + δK−1.Unlike PGD-based training, which targets one estimated nearby maximum, FreeLB minimizes maximum risk at each ascent step with almost no overhead.
  • Generalization and robustness: FreeLB enforces invariance to multiple adversaries across different norm constraints, and empirically produces higher embedding-space robustness and invariance than PGD.The reported robustness measure is the maximum loss increase near X; smaller increases are observed for FreeLB-trained models.

4 EXPERIMENTS

Experiments across GLUE, ARC, and CommonsenseQA evaluate FreeLB against adversarial-training baselines and show improvements across language understanding and reasoning tasks. Additional analyses examine robustness, dropout-mask reuse, comparisons with YOPO, and transfer to ALBERT.

  • GLUE: FreeLB is consistently better than PGD and FreeAT on GLUE dev sets under matched step sizes and ascent-step counts.The evaluation covers RoBERTa-large results across GLUE tasks.
  • GLUE: 1.1 points lift BERT-base’s overall GLUE score from 78.3 to 79.4, while RoBERTa-large rises from 88.5 to 88.8.These are evaluation-server overall scores after finetuning with FreeLB.
  • ARC: 84.81%/65.36% test accuracy on ARC-Easy/ARC-Challenge improves over 84.13%/64.44% without FreeLB.The models are first finetuned on RACE and then on the ARC datasets.
  • ARC: 85.44% ARC-Easy and 67.75% ARC-Challenge test accuracy outperform the best official leaderboard submissions for a single model.This result uses additional-task multi-task learning before FreeLB finetuning.
  • CommonsenseQA: 78.81% dev accuracy on CommonsenseQA is a 1.25% absolute gain over the RoBERTa-large model without FreeLB, and single-model test accuracy reaches 72.2%.A 20-model ensemble reaches 73.1% and ranks first among submissions without ConceptNet.
  • Ablation study and analysis: Reusing the same dropout mask during ascent produces larger improvements over naturally trained models and is enabled for RoBERTa experiments.The comparison is reported in Table 4.
  • Ablation study and analysis: FreeLB outperforms YOPO on all three evaluated datasets, while exhaustive hyperparameter search for both methods remains future work.The comparison adjusts YOPO’s step size to equalize total adversary updates.

5 CONCLUSION

FreeLB improves natural language understanding by perturbing continuous word embeddings and efficiently minimizing adversarial risk. Experiments report stronger embedding-space robustness and generalization, while computational overhead remains a limitation.

  • FreeLB perturbs continuous word embeddings with a gradient method and efficiently minimizes the resulting adversarial risk.
  • FreeLB boosts BERT, RoBERTa, and ALBERT across several datasets, achieving new state-of-the-art results on GLUE and ARC benchmarks.
  • Empirical results associate FreeLB with higher embedding-space robustness and better generalization than natural training.
  • Adversarial training still incurs significant overhead compared with vanilla SGD.

A.1 PROBLEM FORMULATIONS

The appendix describes task-specific problem formulations for ranking-based tasks and implementation adjustments for WNLI and QNLI. These choices align FreeLB inputs and training procedures with the task formats.

  • For ranking-loss tasks such as ARC, CommonsenseQA, WNLI, and QNLI, perturbations are added to concatenated embeddings of all question-answer pairs.
  • WNLI training uses the WSC data provided by Liu et al., while its annotated test set is reordered to match the GLUE dataset.
  • QNLI is formulated as a pairwise ranking problem following Liu et al.

A.2 HYPER-PARAMETERS

FreeLB introduces three additional hyperparameters beyond the base training setup: perturbation step size, maximum perturbation, and number of steps. The appendix lists task-specific settings for GLUE.

  • FreeLB adds step size α, maximum perturbation ǫ, and number of steps m as additional hyperparameters.
  • Other hyperparameters follow RoBERTa settings or are searched over comparable intervals, with MRPC using a learning rate of 5 × 10^-6.
  • For WSC/WNLI, the best listed combination is ǫ = 1e −2, α = 5e −3, and m = 2.

B VARIANCE OF MAXIMUM INCREMENT OF LOSS

Table 8 reports the median and standard deviation of maximum loss increases around correctly classified RoBERTa-Large development samples. The analysis compares naturally trained and adversarially trained models across three datasets.

  • Table 8 reports median and standard deviation for the maximum increase in loss around dev-set samples.
  • The comparison covers RoBERTa-Large models finetuned with different methods, including vanilla naturally trained models.
  • The analysis retains only clean samples correctly classified by every evaluated model, yielding 227 RTE, 850 CoLA, and 355 MRPC samples.

C ADDITIONAL RESULTS FOR ABLATION STUDIES

The additional ablation results compare FreeLB with YOPO across selected GLUE tasks and examine the effect of YOPO’s extra shallow-layer updates. Current results indicate that these updates hurt performance while adding computation.

  • Scope of additional results: Complete comparisons with YOPO and without variational dropout on each GLUE task are planned for a subsequent revision.The current passage presents these results as complementary to Table 4.
  • Comparison setup: The comparison covers STS-B, SST-2, QNLI, QQP, and MNLI dev-set scores from the GLUE benchmark.Scores are summarized by median and standard deviation across five runs with fixed hyperparameters and different random seeds, except YOPO on QQP, which uses four runs.
  • Method variants: FreeLB-m denotes FreeLB with m ascent steps, while YOPO-3-n denotes YOPO with n shallow-layer updates.For YOPO-m-n, the adversary uses step size α, making its effective step size nα.
  • Ablation finding: Extra shallow-layer updates advocated by YOPO consistently deteriorate performance while introducing additional computations.The authors report no need for these updates based on the current comparison results.
Loading 1909.11764v5…