PaperlayerView exact revision on arXiv

Source-linked AI summary

Making Pre-trained Language Models Better Few-shot Learners

Tianyu Gao, Adam Fisch, Danqi Chen

arXiv:2012.15723v2cs.CLcs.LG

TL;DR

Few-shot learning with GPT-3 is powerful but difficult to deploy because of its scale, motivating practical methods for smaller models and limited annotations. LM-BFF combines prompt-based fine-tuning with automated prompt generation and selected demonstrations, achieving up to 30% absolute and 11% average improvement over standard fine-tuning across evaluated tasks.

  • Problem

    GPT-3's 175B parameters make its demonstrated few-shot approach challenging to use, motivating few-shot learning with moderately sized language models and few annotations.

  • Method

    LM-BFF fine-tunes language models with automatically searched prompts and selected task demonstrations incorporated into each input context.

  • Results

    30% absolute improvement at most and 11% on average are obtained over standard fine-tuning across the evaluated few-shot tasks.

  • Takeaways & Limitations

    LM-BFF is presented as a strong task-agnostic few-shot method requiring minimal task resources and domain expertise.

  • Takeaways & Limitations

    LM-BFF still substantially lags behind fine-tuning with thousands of examples, especially on harder tasks, and its results have high variance.

Abstract

from arXiv · show

The recent GPT-3 model (Brown et al., 2020) achieves remarkable few-shot performance solely by leveraging a natural-language prompt and a few task demonstrations as input context. Inspired by their findings, we study few-shot learning in a more practical scenario, where we use smaller language models for which fine-tuning is computationally efficient. We present LM-BFF--better few-shot fine-tuning of language models--a suite of simple and complementary techniques for fine-tuning language models on a small number of annotated examples. Our approach includes (1) prompt-based fine-tuning together with a novel pipeline for automating prompt generation; and (2) a refined strategy for dynamically and selectively incorporating demonstrations into each context. Finally, we present a systematic evaluation for analyzing few-shot performance on a range of NLP tasks, including classification and regression. Our experiments demonstrate that our methods combine to dramatically outperform standard fine-tuning procedures in this low resource setting, achieving up to 30% absolute improvement, and 11% on average across all tasks. Our approach makes minimal assumptions on task resources and domain expertise, and hence constitutes a strong task-agnostic method for few-shot learning.

1 Introduction

LM-BFF adapts prompt-based few-shot learning to moderately sized language models by automating prompt design and selectively incorporating demonstrations. Across evaluated tasks, these techniques substantially outperform standard fine-tuning with very few examples.

  • Motivation: The proposed setting fine-tunes moderately sized language models using only a small number of annotated examples.The setting is intended to be computationally practical and realistic for acquiring limited annotations.
  • Approach: Prompt-based prediction reformulates downstream tasks as masked language modeling, using task-specific templates and label words to generate textual responses.Manual prompt construction can require domain expertise and may be suboptimal.
  • Approach: LM-BFF automatically searches label words and generates templates from few-shot training data, reducing human involvement in prompt design.Its pipeline includes pruned search over label words and template generation with T5.
  • Approach: Selected demonstrations provide additional context through a refined sampling strategy that pairs inputs with similar examples for more discriminative comparisons.This strategy is designed to avoid the limitations of randomly mixing many demonstrations into long contexts.
  • Results: 30% absolute improvement over standard fine-tuning is achieved at most, with 11% average gains across 8 single-sentence and 7 sentence-pair tasks.The evaluation examines few-shot performance across a broad set of classification and related NLP tasks.

2 Related Work

LM-BFF builds on prompt-based learning and distinguishes itself through automatic search over templates and label words using only a few annotations. It also targets downstream fine-tuning rather than knowledge extraction or settings requiring additional resources.

  • Language model prompting: LM-BFF follows prompt-based learning methods developed in the GPT series and studied for fine-tuning by PET.The paper applies prompting to few-shot supervised fine-tuning with a small annotated dataset.
  • Language model prompting: Unlike semi-supervised prompting work, LM-BFF uses only a few annotated examples and additionally explores automatically generated prompts and fine-tuning with demonstrations.Its evaluation is presented as a more rigorous few-shot framework.
  • Automatic prompt search: LM-BFF searches over both templates and label words, and can match or outperform manually selected prompts.The method aims to provide general-purpose search relying only on a few annotations.
  • Automatic prompt search: Earlier automatic prompt approaches either operate in limited domains or require many examples for gradient-guided search.LM-BFF instead targets general-purpose search from few annotations.
  • Fine-tuning: Compared with fine-tuning studies centered on optimization and regularization, LM-BFF focuses on prompt-based fine-tuning in an extreme few-shot setting.The paper anticipates that stabilization methods are complementary to its approach.
  • Few-shot learning: LM-BFF assumes only a few annotated examples and a pre-trained language model, unlike paradigms supplying unlabeled examples, auxiliary tasks, or intermediate training.The stated goal is to assess few-shot learning without these additional advantages.

3 Problem Setup

The study defines few-shot task learning as fine-tuning a pre-trained language model with K examples per class and evaluating generalization to an unseen test set. It evaluates this setting across diverse sentence and sentence-pair tasks while accounting for split and hyperparameter instability.

  • Task formulation: The setup provides a pre-trained language model, a task label space, and K training examples per class, totaling Ktot = K × |Y| examples.The goal is task-agnostic generalization to an unseen test set.
  • Task formulation: Model selection and hyperparameter tuning use a development set the same size as the few-shot training set.Using a larger development set would confer a significant advantage and undermine the limited-data objective.
  • Task formulation: For regression, the data are partitioned into two classes according to whether values are above or below the median.This convention defines the few-shot class structure for regression experiments.
  • Task formulation: The experiments use RoBERTa-large and generally set K = 16.These are the stated default model and per-class example count unless otherwise specified.
  • Evaluation datasets: The evaluation covers 8 single-sentence and 7 sentence-pair English tasks, including GLUE, SNLI, and six other sentence-classification datasets.Single-sentence tasks predict from one sentence, while sentence-pair tasks predict the relationship between two sentences.
  • Evaluation protocol: Performance is averaged over 5 randomly sampled training and development splits to measure few-shot performance more robustly.Multiple hyperparameters are swept for each split, with the best development-set setting selected.

4 Prompt-based Fine-tuning

Prompt-based fine-tuning reformulates classification and regression as masked language modeling, reusing pretrained vocabulary parameters instead of adding a randomly initialized task head. The section also motivates careful prompt construction and automatic search because templates and label words strongly affect few-shot performance.

  • 4 Prompt-based Fine-tuning: Standard fine-tuning adds randomly initialized task-head parameters, making it difficult to learn effectively from as few as 32 annotated examples.For RoBERTa-large, even binary classification introduces 2,048 new parameters.
  • 4 Prompt-based Fine-tuning: Prompt-based fine-tuning instead asks the language model to autocomplete a natural-language prompt containing a masked position.For sentiment classification, the model chooses between label words such as “great” and “terrible.”
  • 4.1 Classification: Classification maps task labels to vocabulary words and converts each input into a masked-language-modeling template with one [MASK] token.The resulting class probabilities are modeled through the masked token’s vocabulary predictions.
  • 4.1 Classification: Prompt-based classification reuses pretrained vocabulary weights, introduces no new parameters, and reduces the gap between pretraining and fine-tuning.The model is fine-tuned with cross-entropy on supervised examples.
  • 4.2 Regression: Regression represents the target interval as interpolation between opposing label words assigned to the lower and upper endpoints.For sentiment in [0, 1], “terrible” and “great” represent the two poles.
  • 4.2 Regression: The regression prediction is the endpoint-weighted mixture y = v_l · p(y_l | x_in) + v_u · p(y_u | x_in).The model fine-tunes the upper-pole probability using KL-divergence against the observed mixture weight.
  • 4.3 Manual prompts: the good and the bad: A prompt consists of a template and label-word mapping, whose manual construction usually requires domain expertise and trial-and-error.Manual choices were based on intuition and formats from previous literature.
  • 4.3 Manual prompts: the good and the bad: Prompt choices substantially affect accuracy: semantically matched label words outperform weaker alternatives, while even small template changes can matter.Prompt search is difficult because the space is large and few examples make overfitting easy.

5 Automatic Prompt Generation

LM-BFF automates prompt construction by searching label-word mappings and generating templates, reducing reliance on manually designed prompts. It uses T5-based candidate generation and validation on development data to select effective prompts.

  • 5 Automatic Prompt Generation: The prompt-generation pipeline searches both label-word mappings and templates to reduce human involvement and improve manually chosen settings.The procedure assumes classification tasks, with an analogous process for regression.
  • 5.1 Automatic selection of label words: For each class, candidate label words are pruned before selecting assignments through zero-shot ranking, fine-tuning, and development-set re-ranking.Pruning addresses the exponential search space and overfitting risks of exhaustive assignment search.
  • 5.2 Automatic generation of templates: T5 generates diverse templates from training examples by filling placeholder spans without requiring a predefined template length.Input examples are converted into T5 inputs, which are used to construct candidate templates.
  • 5.2 Automatic generation of templates: Template decoding selects outputs that maximize their aggregate T5 probability across the training examples.The objective decomposes over template tokens, whose sequence forms the generated template.
  • 5.2 Automatic generation of templates: Beam search with a wide beam produces many diverse template candidates for subsequent evaluation.A beam width such as 100 is used to obtain a large candidate set cheaply.
  • 5.2 Automatic generation of templates: Generated templates are fine-tuned separately and evaluated on development data, either selecting the best single template or ensembling the top candidates.The small training set keeps this per-template evaluation fast and fully automated.
  • 5.2 Automatic generation of templates: For single-sentence tasks, label words may appear before or after the input, whereas sentence-pair tasks place them between the sentences.This placement choice is reported as an implementation detail of template generation.

6 Fine-tuning with Demonstrations

The paper refines demonstration-based fine-tuning by replacing random example selection with dynamically sampled, semantically similar demonstrations. This addresses context-length limits and the difficulty smaller models have using long, mixed-class contexts.

  • 6 Fine-tuning with Demonstrations: Randomly concatenating up to 32 demonstrations is limited by context length and can create long, difficult-to-use contexts for smaller language models.GPT-3 uses a context size of 2,048 tokens, whereas models such as RoBERTa commonly use 512.
  • 6 Fine-tuning with Demonstrations: The proposed strategy samples demonstrations dynamically for each input during both training and inference.At test time, predictions are ensembled across multiple sampled demonstration sets.
  • 6 Fine-tuning with Demonstrations: Demonstrations and query inputs are drawn from the training set during training, while test-time demonstration sets remain sampled from that same training set.This sampling procedure is used to form multiple contexts for each query.
  • 6 Fine-tuning with Demonstrations: Selecting demonstrations is crucial because examples that differ substantially from one another or from the query can obscure meaningful patterns.Such examples may cause the model to ignore or become confused by the additional context.
  • 6 Fine-tuning with Demonstrations: The study also reports that sampling multiple examples per class did not improve performance.This observation is given as an additional sampling result.
  • 6 Fine-tuning with Demonstrations: Table 3 reports RoBERTa-large results across majority, fine-tuning, manual-prompt, automatic-prompt, and GPT-3-style in-context-learning conditions.Results are means and standard deviations over five randomly sampled splits, with K = 16 examples per class in few-shot experiments unless otherwise noted.
  • 6 Fine-tuning with Demonstrations: The method uses SBERT embeddings and cosine similarity to rank training instances within each class, sampling only from the top 50%.For sentence-pair tasks, the two sentences are concatenated before embedding.

7 Experiments

Experiments compare LM-BFF with standard fine-tuning and evaluate automatic prompts, prompt ensembles, and demonstration-sampling strategies across few-shot NLP tasks. The combined approach improves performance substantially, while selective demonstrations and larger prompt ensembles provide additional gains.

  • Main results: Across the main experiments, automatic template search is compared with manual prompts and several fine-tuning and prompting baselines using RoBERTa-large.The evaluation includes standard fine-tuning, full-data fine-tuning, majority-class prediction, prompt-based zero-shot prediction, and GPT-3-style in-context learning with 32 demonstrations.
  • Single-prompt results: Prompt-based fine-tuning greatly outperforms standard fine-tuning, whether using manual or automatically generated prompts.Automatically searched templates can match or exceed manual prompts, especially on TREC, QNLI, and MRPC.
  • Combined approach: 30% gain on SNLI and 11% gain on average are achieved over standard fine-tuning by combining automatically searched templates with sampled demonstration sets.Using demonstrations in context produces consistent gains on a majority of tasks.
  • Ensemble results: Ensembling multiple templates always improves performance, and equally sized ensembles of automatic templates match or outperform PET’s manual-prompt ensembles.Increasing the number of automatic templates brings further gains on MNLI and RTE.
  • Demonstration sampling: Selective demonstration sampling outperforms uniform sampling, indicating that choosing similar examples is important for incorporating demonstrations in context.The comparison uses SBERT and a sentence encoder based on mean-pooled RoBERTa-large representations.
  • Effect of training-set size: With 32 total examples, LM-BFF nearly saturates performance on SST-2 and remains ahead of standard fine-tuning on SNLI until both converge around K = 256.LM-BFF is comparable to full-dataset standard fine-tuning on SST-2 despite using far fewer examples.

8 Discussion

LM-BFF improves few-shot fine-tuning but remains limited by performance gaps, variance, prompt-search constraints, and task-formulation requirements.

  • Limitations: LM-BFF still substantially lags behind fine-tuning with thousands of examples, especially on harder tasks.Its few-shot results also exhibit high variance, as do standard fine-tuning results.
  • Limitations: Automatic prompt generation remains difficult to expand or generalize from approximately 32 examples.The search still relies partly on manual templates or label words, biasing it toward previously imagined regions.
  • Empirical pattern: For lower K, LM-BFF consistently outperforms standard fine-tuning.The comparison varies K, defined as the number of instances per class.
  • Limitations: LM-BFF favors tasks that fit fill-in-the-blank formulations, use relatively short inputs, and have few output classes.The formulation constraint is more fundamental for tasks such as structured prediction.

9 Conclusion

The paper concludes that LM-BFF combines prompt-based fine-tuning, automatic prompt search, and selected demonstrations for few-shot learning. It reports improvements over vanilla fine-tuning and notes that development-set size affects performance.

  • Conclusion: LM-BFF combines automatically searched prompt-based fine-tuning with selected task demonstrations in the input context.The approach is presented as a set of simple techniques for fine-tuning language models with few examples.
  • Conclusion: LM-BFF outperforms vanilla fine-tuning by up to 30% and 11% on average.These are the paper's reported aggregate improvement figures.
  • Conclusion: Larger development sets lead to better performance in the examined few-shot setting.The experiments therefore use development sets with the same size as the training sets.

B Datasets

The evaluation covers sentence-level and sentence-pair datasets from SNLI and GLUE, including classification and regression tasks, using few-shot subsets.

  • Datasets: The evaluation uses SNLI and GLUE datasets including SST-2, CoLA, MNLI, QNLI, RTE, MRPC, QQP, and STS-B.STS-B is treated as a real-valued regression task, while the other listed tasks are classification datasets.
  • Datasets: Few-shot experiments sample Dtrain and Ddev with K × |Y| examples from the original training set.Here, |Y| denotes the number of classes, except for the real-valued STS-B regression task.

C Experimental Details

The experiments specify optimization, prompt-search, demonstration, dataset, model-segment, and implementation settings for few-shot evaluation.

  • Optimization: Grid search uses learning rates of 1e-5, 2e-5, and 5e-5 with batch sizes of 2, 4, and 8.Early stopping validates every 100 steps during 1,000-step trials and selects the best checkpoint.
  • Prompt search: Automatic prompt generation uses T5-3B, while label-word search generally uses k = 100 candidates.SST-5 uses k = 30, and automatic-search trials use batch size 8 with learning rate 1e-5.
  • Implementation: Prompt implementation adjusts spacing, capitalization, and punctuation so inputs more closely match pre-training distributions.These transformations apply when sentences are concatenated with prefixes or appended punctuation.
  • Demonstrations: Inference with demonstrations averages predicted class log probabilities across 16 sampled demonstration sets.Further increasing the number of samples or using maximum-confidence aggregation did not yield substantial improvement.
  • Prompt search: T5-1.0 and T5-1.1 produced no significant difference in generated templates.The T5-1.0 checkpoint includes unsupervised and downstream-task data.
  • Datasets: The benchmark includes sentence-level and sentence-pair datasets with few-shot subsets of K × |Y| examples.The dataset table records class counts and average input lengths; STS-B is real-valued regression over [0, 5].
  • Model configurations: For BERT with demonstrations, 2-seg works best, while RoBERTa-large performs much better overall and avoids segment-embedding tuning.The segment strategies compare one shared segment, input-versus-demonstration segments, and separate embeddings per sentence.
  • Prompt analysis: Automatically generated templates are generally grammatical, but generated label words can be counterintuitive for some tasks.Automatic templates are evaluated with manual label words, and automatic label words with manual templates.
Loading 2012.15723v2…