PaperlayerView exact revision on arXiv

Source-linked AI summary

Finetuned Language Models Are Zero-Shot Learners

Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, Quoc V. Le

arXiv:2109.01652v5cs.CL

TL;DR

Large language models are strong few-shot learners but perform less well without examples. This paper instruction-tunes a 137B-parameter model on over 60 instructed NLP tasks, and FLAN improves zero-shot performance on unseen tasks, surpassing zero-shot GPT-3 on 20 of 25 datasets.

  • Problem

    Large language models perform substantially worse in zero-shot than few-shot settings, limiting performance on tasks such as reading comprehension, question answering, and natural language inference.

  • Method

    The authors instruction-tune a 137B-parameter language model on over 60 NLP datasets while holding out entire task clusters to evaluate unseen tasks.

  • Results

    FLAN outperforms zero-shot GPT-3 on 20 of 25 datasets, while unseen-task performance improves with more task clusters and sufficient model scale.

  • Takeaways & Limitations

    Instruction tuning shows that labeled data can improve large language models’ cross-task generalization and support instruction-following on unseen tasks.

  • Takeaways & Limitations

    The study’s 137B-parameter FLAN model makes deployment costly.

Abstract

from arXiv · show

This paper explores a simple method for improving the zero-shot learning abilities of language models. We show that instruction tuning -- finetuning language models on a collection of tasks described via instructions -- substantially improves zero-shot performance on unseen tasks. We take a 137B parameter pretrained language model and instruction-tune it on over 60 NLP tasks verbalized via natural language instruction templates. We evaluate this instruction-tuned model, which we call FLAN, on unseen task types. FLAN substantially improves the performance of its unmodified counterpart and surpasses zero-shot 175B GPT-3 on 20 of 25 tasks that we evaluate. FLAN even outperforms few-shot GPT-3 by a large margin on ANLI, RTE, BoolQ, AI2-ARC, OpenbookQA, and StoryCloze. Ablation studies reveal that number of finetuning datasets, model scale, and natural language instructions are key to the success of instruction tuning.

1 INTRODUCTION

The paper introduces instruction tuning to improve large language models’ zero-shot performance by finetuning them on tasks expressed through natural-language instructions. FLAN substantially improves its 137B-parameter base model and outperforms GPT-3 zero-shot on most evaluated datasets, with ablations identifying key factors behind the gains.

  • Method: Instruction tuning finetunes a 137B-parameter pretrained language model on more than 60 NLP tasks described through natural-language instructions.The approach uses task descriptions such as sentiment classification and translation instructions.
  • Evaluation: FLAN is evaluated zero-shot on unseen task types by holding out each task-type cluster during instruction tuning.For example, natural language inference is evaluated after training on other task types, including commonsense reasoning, translation, and sentiment analysis.
  • Results: FLAN substantially improves the base 137B-parameter model’s zero-shot performance and surpasses 175B-parameter GPT-3 zero-shot on 20 of 25 evaluated datasets.It also beats GPT-3 few-shot by a large margin on ANLI, RTE, BoolQ, AI2-ARC, OpenbookQA, and StoryCloze.
  • Ablations and interpretation: Increasing the number of task clusters improves performance on unseen tasks, while instruction tuning combines finetuning supervision with inference-time text interactions.The paper presents instruction tuning as combining aspects of pretrain–finetune and prompting paradigms.

2 FLAN: INSTRUCTION TUNING IMPROVES ZERO-SHOT LEARNING

FLAN instruction-tunes a 137B-parameter language model on diverse tasks expressed through natural-language instructions, evaluating zero-shot generalization to task clusters withheld from tuning. The method uses conservative cluster-level splits, preserves free-text generation, and applies special handling to classification outputs.

  • 2 FLAN: INSTRUCTION TUNING IMPROVES ZERO-SHOT LEARNING: Instruction tuning teaches models to follow task instructions and generalize to unseen tasks by holding out each task-type cluster during evaluation.Training uses all remaining clusters while evaluating on the held-out cluster.
  • 2.1 TASKS & TEMPLATES: 62 publicly available text datasets spanning language understanding and generation are aggregated into one instruction-tuning mixture.The datasets come from Tensorflow Datasets and are organized into twelve task clusters.
  • 2.1 TASKS & TEMPLATES: Ten manually composed natural-language templates describe each dataset, with up to three additional templates reversing the task to increase diversity.For example, sentiment classification can include templates that ask the model to generate a movie review.
  • 2.2 EVALUATION SPLITS: A dataset counts as unseen only when no dataset from any task cluster to which it belongs appeared during instruction tuning.This cluster-based definition is more conservative than holding out only the same dataset.
  • 2.3 CLASSIFICATION WITH OPTIONS: FLAN is a 137B-parameter decoder-only model that directly produces free text for generation tasks, while classification requires choosing among specified output options.The model is the instruction-tuned version of LaMDA-PT; classification uses an option-based procedure to address variation in ways of expressing answers.

3 RESULTS

FLAN substantially improves zero-shot performance across instruction-friendly task clusters, outperforming major pretrained baselines on many unseen datasets. Gains are weaker on language-modeling-style tasks, where instruction tuning is often redundant.

  • Overall comparisons: 20 of 25 datasets: zero-shot FLAN outperforms zero-shot GPT-3, while also surpassing GPT-3’s few-shot performance on 10 datasets.With the best dev template, FLAN also outperforms zero-shot GLaM on 13 of 19 available datasets and one-shot GLaM on 11 of 19 datasets.
  • Overall findings: Instruction tuning is most effective for naturally verbalized tasks, including NLI, QA, translation, and struct-to-text, but less effective for language-modeling-formulated tasks.The latter include commonsense reasoning and coreference resolution tasks formatted as incomplete sentence or paragraph completion.
  • Natural language inference: On five NLI datasets, FLAN outperforms all baselines by a large margin, aided by phrasing NLI as a natural-language question rather than sentence continuation.The paper suggests GPT-3 struggles because NLI examples are unlikely to have appeared naturally in unsupervised training data.
  • Reading comprehension and closed-book QA: FLAN outperforms GPT-3 on all four closed-book QA datasets and beats GLaM on ARC-e and ARC-c, while slightly trailing it on NQ and TQA.For reading comprehension, FLAN outperforms baselines on MultiRC and OBQA and beats GPT-3 by a large margin on BoolQ.
  • Translation: FLAN outperforms zero-shot GPT-3 on all six translation evaluations but underperforms few-shot GPT-3 in most cases.Translation into English is strong, while translation from English into other languages is relatively weaker.
  • Additional tasks: 3 of 7 tasks: FLAN outperforms LaMDA-PT on commonsense reasoning and coreference resolution tasks formulated as sentence completions.This negative result indicates that instruction tuning does not improve many language-modeling tasks.

4 ABLATION STUDIES & FURTHER ANALYSIS

Ablations show that instruction tuning benefits zero-shot generalization most with more task clusters, large models, and natural-language instructions. Few-shot exemplars and prompt tuning further improve FLAN’s performance and robustness, especially in complex or low-resource settings.

  • Task clusters and datasets: Average performance on three held-out clusters improves as additional instruction-tuning clusters and tasks are added, without apparent saturation across seven clusters.The exception is the sentiment-analysis cluster.
  • Model scale: Instruction tuning substantially improves held-out-task performance for models around 100B parameters but hurts generalization for 8B and smaller models.A proposed explanation is that small models devote their capacity to learning the instruction-tuning task mixture.
  • Natural-language instructions: Both instruction-free finetuning configurations perform substantially worse than FLAN, indicating that natural-language instructions are important during training.The configurations remove instructions entirely or replace them with dataset names.
  • Few-shot inference: Few-shot exemplars improve every task cluster over zero-shot FLAN and reduce template-level standard deviation, with especially strong effects on complex output spaces.The highlighted task types are struct to text, translation, and closed-book QA.
  • Prompt tuning: More than 10% improvement is achieved in many low-resource prompt-tuning cases with FLAN over LaMDA-PT, while FLAN performs better in every tested scenario.The experiments use both fully supervised training sets and only 32 training examples.

5 RELATED WORK

The paper situates its approach within zero-shot learning, prompting, multi-task learning, and language models for NLP, while focusing on two narrower related-work areas. It contrasts QA-based task formulations and emerging instruction-following research with its emphasis on zero-shot learning and pretrained-model knowledge.

  • Broad research areas: Related work spans zero-shot learning, prompting, multi-task learning, and language models for NLP applications.The paper discusses these broad areas in an extended related-work section in Appendix D.
  • QA-based task formulation: QA-based task formulation casts NLP tasks as question answering over context, but prior methods mainly target multi-task learning rather than zero-shot learning.The paper also notes that these methods are generally not motivated by using existing knowledge in pretrained language models.
  • Instruction-following research: Mishra et al. (2021) finetune 140M-parameter BART on instructions with few-shot exemplars and evaluate few-shot abilities on unseen tasks.The paper describes this result as similar to its few-shot instruction-tuning result from §4.4.

6 DISCUSSION

Instruction tuning improves FLAN’s performance on unseen tasks relative to an untuned model and surpasses zero-shot GPT-3 on most evaluated tasks. The study also suggests labeled data can support large language models across many unseen tasks, while noting limitations in task clustering, instruction length, and possible pretraining overlap.

  • Main findings: FLAN improves performance against an untuned model and surpasses zero-shot GPT-3 on the majority of evaluated tasks.The paper frames instruction tuning as combining aspects of pretrain–finetune and prompting paradigms.
  • Implications: Instruction tuning shows how labeled data can help large language models perform many unseen tasks, with implications for specialist–generalist model tradeoffs.The study contrasts specialist models, which target one task, with generalist models, which handle many tasks.
  • Limitations: Task-cluster assignment is subjective despite efforts to follow accepted categorizations in the literature.This subjectivity is identified as a limitation of the study.
  • Limitations: The study explores only relatively short instructions, typically a single sentence, and individual examples may have appeared in pretraining data.The potential overlap with pretraining data is noted as an evaluation limitation.

7 CONCLUSIONS

The paper presents instruction tuning as a simple method for improving large language models’ zero-shot performance based purely on instructions. FLAN compares favorably with GPT-3 and highlights the potential of instruction-following language models at scale.

  • Conclusions: Instruction tuning improves large language models’ ability to perform zero-shot tasks based purely on instructions.The paper frames this as a simple method applicable to language models at scale.
  • Conclusions: FLAN compares favorably against GPT-3, supporting the potential for large language models to follow instructions.This comparison motivates further study of instruction-based NLP and zero-shot learning.
  • Conclusions: The authors call for further research on instruction-based NLP, zero-shot learning, and using labeled data to improve large language models.These areas are presented as directions encouraged by the paper’s findings.

ETHICAL CONSIDERATIONS

The paper notes that language-model risks and potential harms are relevant, while emphasizing two additional ethical considerations specific to studying instruction tuning and zero-shot performance on unseen tasks.

  • ETHICAL CONSIDERATIONS: The authors distinguish their empirical study of instruction tuning from developing a pretrained language model and therefore highlight two additional ethical considerations.The broader risks and potential harms of language models are discussed in prior work cited by the paper.

ENVIRONMENTAL CONSIDERATIONS · R Bar Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan

The paper reports substantial pretraining energy and carbon costs, while noting that FLAN’s additional instruction-tuning cost is comparatively smaller. The merged material also includes a bibliography subsection listing related work.

  • ENVIRONMENTAL CONSIDERATIONS: 451 MWh and 26 tCO2e were the reported energy cost and carbon footprint for the pretrained models.These models are the same pretrained language models used by Austin et al. (2021).
  • ENVIRONMENTAL CONSIDERATIONS: Less than 2% of pretraining steps were additional instruction-tuning gradient-steps for FLAN, making its estimated extra energy cost comparatively smaller.The comparison concerns instruction-tuning gradient-steps relative to pretraining steps.
  • R Bar Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan: The subsequent subsection consists of related-work references spanning textual entailment, question answering, multilingual modeling, prompting, summarization, and instruction following.The entries include datasets, methods, benchmarks, and prior language-modeling work.

A ADDITIONAL RESULTS … B.3 FURTHER ANALYSIS: INSTRUCTION TUNING FACILITATES PROMPT TUNING

The additional results report evaluation coverage and methodological caveats, while further analyses show that expanding datasets per task cluster improves held-out performance far more than adding templates. The appendices also provide per-cluster and per-dataset ablation results, with broader task types identified as future work.

  • A ADDITIONAL RESULTS: Results cover translation, struct-to-text, and eight NLU task clusters, with FLAN evaluated using up to ten instruction templates.The reported tables distinguish translation and struct-to-text results from the eight NLU clusters.
  • A ADDITIONAL RESULTS: FLAN uses greedy search for generative tasks, while LaMDA-PT uses GPT-3-optimized templates without model-specific prompt engineering.FLAN performance is reported both for the best of up to ten templates and for the template best on the development set.
  • A ADDITIONAL RESULTS: GPT-3 zero-shot results for DROP and SQuADv2 are not directly comparable because those evaluations use exemplars from the same passage.The paper marks these results with the † symbol.
  • A ADDITIONAL RESULTS: Summarization evaluation is deferred because most summarization datasets exceed FLAN’s 1024-token input length.Although a summarization task cluster is used during instruction tuning, the paper leaves summarization evaluation for future work.
  • B.1 DATASETS PER TASK CLUSTER & TEMPLATES PER DATASET: 10%: using more datasets per cluster improves held-out performance by almost 10% on average across NLI, commonsense reasoning, and closed-book QA.The held-out clusters are trained separately from seven remaining task clusters.
  • B.1 DATASETS PER TASK CLUSTER & TEMPLATES PER DATASET: Adding more templates per dataset has a comparatively negligible effect, disappearing when there are four tasks per cluster.The small template effect persists despite the motivation that ten templates per task would reduce overfitting to individual templates.
  • B.2 ROLE OF INSTRUCTIONS DURING FINETUNING: Per-cluster results for the instruction-role ablation are reported in Table 3.The passage identifies the table as the source for the ablation’s per-cluster results but does not provide its values here.
  • B.3 FURTHER ANALYSIS: INSTRUCTION TUNING FACILITATES PROMPT TUNING: Per-dataset results for the prompt-tuning analysis are reported in Table 4, while future work could extend beyond classification to summarization or question answering.The passage also suggests finetuning the model using supervised datasets.

C DATA CONTAMINATION ANALYSIS · D EXTENDED RELATED WORK

The contamination analysis tested whether FLAN’s zero-shot results were inflated by pretraining overlap. Clean-set evaluations showed no consistent performance disadvantage, while highly contaminated datasets mostly overlapped in background contexts rather than answer-bearing content.

  • C DATA CONTAMINATION ANALYSIS: FLAN’s post-hoc analysis evaluated whether benchmark examples had appeared verbatim in its more-than-2-trillion-token pretraining corpus.This directly addressed the concern that pretraining overlap could inflate purported zero-shot performance.
  • C DATA CONTAMINATION ANALYSIS: The procedure removed examples sharing any dataset-specific roughly 13-gram overlap with pretraining data, then compared clean-subset and original-dataset performance.The setup followed Brown et al. (2020), using the same n per dataset and splitting on spaces.
  • C DATA CONTAMINATION ANALYSIS: Across datasets, clean-set evaluation showed no correlation with worse performance than evaluation on the total dataset.Datasets with fewer clean examples also exhibited higher variance in percent performance change.
  • C DATA CONTAMINATION ANALYSIS: DROP and SQuADv2 had almost total overlap, but 99.6% of DROP and 97.2% of SQuADv2 overlaps occurred only in example contexts.Overlap occurred in both question and answer for only 5 of 11,153 SQuADv2 evaluation examples, and never for DROP.
  • C DATA CONTAMINATION ANALYSIS: ANLI R1 and R2 also had almost complete contamination, with overlaps concentrated in contexts at 97.3% for R1 and 98.2% for R2.The authors attribute the higher contamination than GPT-3 potentially to a more-recent Wikipedia version in FLAN’s pretraining corpus.
  • C DATA CONTAMINATION ANALYSIS: Only ReCoRD and PIQA had clean-subset performance more than 1% below overall evaluation performance, plausibly because complete-sentence exposure aids language-modeling tasks.For PIQA, both the goal and solution overlapped in 93 examples.
  • C DATA CONTAMINATION ANALYSIS: The analysis remains limited by conservative n-gram matching, possible distribution differences between clean and overall subsets, and the lack of established contamination-detection best practices.The authors also note that the pretraining corpus is almost five times larger than GPT-3’s, complicating comparisons.

D.1 LANGUAGE MODELS AND MULTI-TASK LEARNING … D.6 INSTRUCTIONS-BASED NLP

The paper situates instruction tuning within multitask learning, zero-shot learning, prompting, and large-language-model finetuning. It distinguishes its focus on instruction-tuning models for unseen tasks from prior work emphasizing task-specific finetuning or prompting strategies.

  • D.1 LANGUAGE MODELS AND MULTI-TASK LEARNING: Instruction tuning is framed as a formulation of multitask learning for language models and NLP applications.The paper connects its approach to established multitask-learning and language-model research.
  • D.2 ZERO-SHOT LEARNING AND META-LEARNING: Zero-shot learning traditionally concerns classifying instances among unseen categories and, in NLP, includes unseen language translation and language modeling.The cited NLP examples extend zero-shot learning beyond classification.
  • D.3 PROMPTING: Instruction tuning shares prompting’s goal of leveraging large language models’ capabilities through inference-time text interactions without separate task-specific checkpoints.The paper also relates instruction tuning to continuous-input prompting and specialized-output prompting.
  • D.4 FINETUNING LARGE LANGUAGE MODELS: Unlike prior finetuning work that evaluates on the same downstream task, this setup studies whether instruction tuning enables performance on unseen tasks.Prior work spans dense models from roughly 100M to 10B parameters, O(100B) models, and sparse mixture-of-experts models exceeding 1T parameters.
  • D.5 MULTI-TASK QUESTION ANSWERING: The paper’s instructions resemble research that unifies NLP tasks by casting them as question answering over a context.McCann et al. formulated ten NLP tasks as question answering with natural-language prompts and reported transfer and zero-shot domain-adaptation gains.
  • D.6 INSTRUCTIONS-BASED NLP: Instructions-based NLP includes task descriptions for few-shot and semi-supervised learning, although some prior approaches finetune a new checkpoint for each downstream task.The cited work uses cloze-style task descriptions to help assign soft labels.
  • D.6 INSTRUCTIONS-BASED NLP: Prior instruction-based studies also finetuned models on many tasks and evaluated unseen tasks, reporting improved unseen-task performance or cross-task few-shot learning.Mishra et al. used instructions and few-shot examples with BART for question answering, classification, and text modification; Ye et al. studied cross-task few-shot learning.
  • D.6 INSTRUCTIONS-BASED NLP: Related work finds that few-shot exemplars and instruction tuning are complementary, while InstructGPT combines finetuning and reinforcement learning guided by human annotations.Min et al. finetuned GPT-2 Large as a few-shot learner, and InstructGPT was preferred by human raters over unmodified GPT-3.

E FREQUENTLY ASKED QUESTIONS … CHANGES TO V2 FROM V1

The paper explains that FLAN uses human-like instruction prompts, while noting failures, scaling artifacts, and task-transfer limitations. It also documents qualitative capabilities and successive revisions, including expanded evaluations and a corrected TriviaQA comparison.

  • E FREQUENTLY ASKED QUESTIONS: FLAN prompts resemble requests to a human, unlike GPT-3 continuation-style prompts and T5 dataset tags that are unsuitable for zero-shot use.Because pretrained models do not reliably follow FLAN-style instructions, LaMDA-PT used established GPT-3 prompts as its baseline.
  • E FREQUENTLY ASKED QUESTIONS: FLAN fails on returning the second word, translates a question instead of answering it, has a 1024-token context limit, and was mostly trained on English data.The paper presents these as limitations despite finding that FLAN responds well to most tasks qualitatively.
  • E FREQUENTLY ASKED QUESTIONS: Instruction tuning focuses on zero-shot cross-task generalization, while prompt tuning results provide promising evidence of positive task transfer in supervised settings.The paper leaves the effect of task mixtures on transfer among seen tasks for future work.
  • E FREQUENTLY ASKED QUESTIONS: Scale does not always improve untuned LaMDA-PT zero-shot performance, particularly for the evaluated reading-comprehension and sentiment-analysis tasks.The authors describe this as an artifact and note that GPT-3 similarly reports lower zero-shot BoolQ and DROP performance from 13B to 175B parameters.
  • F QUALITATIVE EXAMPLES: Qualitative examples show FLAN handling flipped sentiment questions, cross-language question answering, rule-based manipulation, recommendations, data generation, query expansion, assisted writing, word formation, and open-ended generation.Figure 22 also presents failure cases involving nth-word extraction and translating a question instead of answering it.
  • CHANGES FROM V4 TO V5: The V4-to-V5 revision replaced main-figure tables with a space-saving zero-shot-focused figure, added GLaM 64B/64E, and moved instruction and prompt-tuning ablations into the main paper.The moved ablations were condensed into fewer figures.
  • CHANGES TO V4 FROM V3: The V3-to-V4 revision added FAQs, qualitative examples, and an instruction-finetuning ablation, while updating related work with newer arXiv manuscripts.The FAQ and qualitative-example material became Appendices E and F, and the instruction ablation became Appendix B.2.

G TASKS AND DATASETS … G.7 TRANSLATION (7 LANGUAGES)

The appendix organizes datasets across language-understanding and generation task clusters, including natural language inference, reading comprehension, commonsense reasoning, translation, and related tasks. It also specifies dataset splits and a 30,000-example training cap to prevent any dataset from dominating finetuning.

  • G TASKS AND DATASETS: The appendix groups datasets into task clusters spanning natural language inference, reading comprehension, commonsense reasoning, sentiment analysis, closed-book QA, paraphrase detection, and coreference resolution.Natural language inference concerns relations between two sentences, while reading comprehension answers questions from passages and commonsense reasoning involves physical or scientific reasoning.
  • G.1 NATURAL LANGUAGE INFERENCE: NLI datasets include ANLI, CB, MNLI, QNLI, SNLI, WNLI, and RTE, covering entailment, non-entailment, and cases that are impossible to say.RTE is binary, whereas ANLI uses entailment, not entailment, or impossible to say and includes adversarial examples collected with a human and model in the loop.
  • G.2 READING COMPREHENSION: Reading-comprehension datasets include BoolQ, DROP, MultiRC, OBQA, SQuADv1, and SQuADv2, while their examples ask questions based on passages or facts.BoolQ uses yes/no questions, MultiRC uses open-ended questions given paragraphs, and OBQA uses 4-way multiple-choice questions based on facts.
  • G.3 COMMONSENSE REASONING: Commonsense reasoning uses COPA, HellaSwag, PiQA, and StoryCloze for causal inference, sentence completion, naive-physics reasoning, and story-ending selection.The datasets require choosing causes or effects, probable sentence endings, solutions to goals, or correct endings to four-sentence stories.
  • 3. WSC273 (Levesque et al., 2012): Additional clusters cover reading comprehension with commonsense, structured-data-to-text generation, translation, summarization, and miscellaneous tasks such as conversational question-answering, word-meaning evaluation, question classification, acceptability, and math.The miscellaneous cluster includes QuAC, CoQA, WiC, TREC, CoLA, and Math questions.
  • G.7 TRANSLATION (7 LANGUAGES): Translation covers seven language directions: En–Fr from WMT’14, En–De, En–Tr, En–Cs, En–Fi, En–Ro, and En–Ru from WMT’16, and En–Es from Paracrawl.Translation is defined as translating text from one language into another, and the evaluation example is from WMT’16 English–German.
  • G TASKS AND DATASETS: 30,000 training examples per dataset was the maximum used so that no dataset dominated the finetuning distribution.The code uses TensorFlow Datasets for loading and processing; labeled TFDS test sets were used when available, otherwise validation sets served as test sets and training data was split into train and dev.
Loading 2109.01652v5…