PaperlayerView exact revision on arXiv

Source-linked AI summary

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning

Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, Adam Roberts

arXiv:2301.13688v2cs.AIcs.CLcs.LG

TL;DR

The paper asks which design decisions make publicly available instruction tuning effective and studies them through controlled ablations of the Flan 2022 Collection. It finds that combining diverse tasks with prompt mixing, enrichment, and balancing improves performance over prior open collections, while Flan-T5 is a stronger and more computationally efficient starting checkpoint for downstream finetuning. The authors release the collection, templates, and methods for public research.

  • Problem

    The paper investigates which methods and data-design choices most effectively improve publicly available instruction tuning.

  • Method

    It combines prior instruction tuning collections with additional datasets and evaluates mixed prompting, task scaling, input inversion, and task balancing through controlled ablations.

  • Results

    Flan 2022 outperforms other public collections by 3–17% on held-out tasks, including 4.2%+ on MMLU and 8.5% on BIG-Bench Hard for equally sized models.

  • Takeaways & Limitations

    Mixed prompting, task enrichment, and balancing are critical techniques, and Flan-T5 provides a stronger, more computationally efficient starting checkpoint for single-task finetuning.

Abstract

from arXiv · show

We study the design decisions of publicly available instruction tuning methods, and break down the development of Flan 2022 (Chung et al., 2022). Through careful ablation studies on the Flan Collection of tasks and methods, we tease apart the effect of design decisions which enable Flan-T5 to outperform prior work by 3-17%+ across evaluation settings. We find task balancing and enrichment techniques are overlooked but critical to effective instruction tuning, and in particular, training with mixed prompt settings (zero-shot, few-shot, and chain-of-thought) actually yields stronger (2%+) performance in all settings. In further experiments, we show Flan-T5 requires less finetuning to converge higher and faster than T5 on single downstream tasks, motivating instruction-tuned models as more computationally-efficient starting checkpoints for new tasks. Finally, to accelerate research on instruction tuning, we make the Flan 2022 collection of datasets, templates, and methods publicly available at https://github.com/google-research/FLAN/tree/main/flan/v2.

1 Introduction

This work analyzes which design choices make publicly available instruction tuning more effective, focusing on the Flan 2022 Collection through controlled comparisons and ablations. It finds that task diversity, prompt mixing, enrichment, and balancing improve performance, while the resulting collection and methods are released for public research.

  • The study evaluates Flan 2022’s instruction tuning methods by ablating individual factors and comparing them with prior work at consistent pretrained model size and checkpoint.
  • 4.2%+ on MMLU and 8.5% on BIG-Bench Hard are reported for equally sized models trained on the Flan 2022 Collection.
  • Mixed zero-shot, few-shot, and chain-of-thought training improves performance across each corresponding prompting setting, with 10% few-shot prompts improving zero-shot results by 2%+.
  • Input inversion, task-source balancing, scaling, and chain-of-thought training are identified as important techniques for effective instruction tuning.
  • 3–17% Held-Out task improvements over existing open source instruction tuning collections result from the evaluated technical choices.
  • The authors open source the Flan 2022 task collection, templates, and methods to support public research.

2 Public Instruction Tuning Collections

Public instruction tuning collections developed from early task and template aggregation toward broader resources combining datasets, prompting formats, and methodological techniques. This work emphasizes accessible, unified resources while focusing on instruction generalization without human feedback.

  • Task counts and examples in the timeline are estimates because works use different definitions of tasks and task categories.
  • Since 2020, instruction tuning collections have rapidly expanded, beginning with resources that aggregate NLP tasks and format them with instructions for unseen-task generalization.
  • The second wave combined more datasets and tasks, added multilingual instruction tuning, and introduced chain-of-thought training prompts.
  • Flan 2022 coalesces most prior collections and methods as a proposed strong starting point for future open source work.
  • The paper focuses on instruction generalization without human feedback because human-feedback datasets are less publicly available and may be model-specific.
  • The authors restrict their scope to open and accessible collections because inaccessible resources limit public analysis and improvement of instruction tuning methods.

3 Flan 2022 Instruction Tuning Experiments

The experiments examine Flan 2022’s design decisions through controlled evaluation of mixed prompting, task scaling, input inversion, and task-mixture balancing. They use a consistent T5-LM XL setup and compare Flan 2022 with existing instruction tuning collections.

  • The Flan 2022 collection combines Flan 2021, P3++, Super-Natural Instructions, and additional reasoning, dialog, and program synthesis datasets.
  • The experiments evaluate four design components: mixed prompting, scaling to 1800+ tasks, input inversion, and balancing task mixtures.
  • P3++ denotes all datasets in the Public Pool of Prompts.
  • Models are finetuned on prefix-language-model-adapted T5-LM XL models with 3B parameters for consistency, unless otherwise stated.
  • The evaluation covers held-in tasks, chain-of-thought tasks, and the MMLU benchmark.

Appendix A.

Flan-T5 XL’s gains come from combining complementary instruction-tuning methods, prompt settings, task scaling, enrichment, and data balancing. These choices improve held-out performance, although task-count scaling depends on how tasks are defined and diversified.

  • Ablation Studies: Flan-T5 XL combines mixture balancing, chain-of-thought data, mixed prompt settings, and input inversion in its full method.The ablation framework evaluates each component individually across Held-In, Chain-of-Thought, and Held-Out tasks.
  • Ablation Studies: Each ablated component improves different metrics: chain-of-thought training helps Chain-of-Thought evaluation, input inversion helps Held-Out evaluation, few-shot training helps few-shot evaluation, and mixture balancing helps all metrics.The method-specific effects align with the evaluation settings targeted by each design choice.
  • Training with Mixed Prompt Settings: Mixed zero-shot and few-shot training improves both prompt settings, with both Held-In and Held-Out performance peaking above single-setting training.Adding as little as 5% few-shot training can improve zero-shot performance, while 10% or more zero-shot data improves few-shot performance.
  • Scaling Small Models to 1.8k+ Tasks: Held-Out performance increases log-linearly with task count and reaches its highest level with all 1836 tasks, while Held-In performance peaks around 200 tasks.Larger models peak later and diminish less on Held-In evaluations.
  • Scaling Small Models to 1.8k+ Tasks: Scaling conclusions assume all tasks are defined and counted equally, so increasing task counts beyond 1800 may not improve returns without task diversity and quality.The paper notes that performance can saturate when too many tasks come from one source.
  • Task Enrichment with Input Inversion: Input inversion is not beneficial for Held-In performance but is strongly beneficial for Held-Out performance.The method reverses supervised input-output pairs to enrich task variety.
  • Balancing Data Sources: Flan 2021 and T0-SF are among the most beneficial mixtures, followed by Super-Natural Instructions and Chain-of-Thought, whereas Dialog and Program Synthesis rank lowest.Task-source ablations inform mixture weighting, but the authors state that more sophisticated search remains possible.

4 Instruction Tuning Enhances Single-Task Finetuning

The paper evaluates Flan-T5 as an intermediate checkpoint before single-task finetuning and finds it improves both final performance and training efficiency relative to direct T5 finetuning.

  • Evaluation setup: Flan-T5 is evaluated as an intermediary checkpoint before finetuning on a single target task.The study compares direct T5 finetuning, Flan-T5 without further finetuning, and further finetuning of Flan-T5.
  • Performance: Flan-T5 offers a Pareto improvement over directly finetuned T5 across both Held-In and Held-Out tasks.With limited finetuning data, Flan-T5 without further finetuning can outperform T5 with task finetuning.
  • Efficiency: Flan-T5 converges faster than T5 during single-task finetuning across each of five Held-Out tasks.Figure 6 compares convergence during single-task finetuning for five tasks held out from Flan finetuning.
  • Efficiency: Flan-T5 reaches higher peak accuracies while converging faster than T5 during single-target finetuning.The paper links these convergence results to potential training-efficiency benefits for instruction-tuned starting checkpoints.
  • Efficiency: Instruction tuning has a one-time computational cost, whereas extensive downstream finetuning can accumulate millions of additional training steps.The paper presents reduced finetuning steps across many tasks as the practical efficiency motivation.

5 Related Work

The related work situates instruction tuning within large-scale pretraining, objective design, parameter-efficient adaptation, and alignment research. It highlights open questions about how pretraining data and instruction tuning jointly affect downstream abilities.

  • Large Language Models: Large language models build on a long-standing paradigm of pretraining general-purpose representations for multiple downstream NLP tasks.The section traces this tradition from early representation learning through large unsupervised pretrained models.
  • Large Language Models: Decoder-only causal Transformers dominated publicly available models larger than 100B as of 2022, partly because of hardware and software support.The passage contrasts this deployment pattern with findings that left-to-right causal modeling can be suboptimal for downstream tasks.
  • Large Language Models: Prior work reports that mixtures of non-sequential pretraining objectives can outperform causal objectives for zero-shot and few-shot downstream prompting.The related work also identifies the relationship between pretraining corpora, instruction tuning, and downstream abilities as under-explored.
  • Instruction Tuning: Parameter-efficient tuning is presented as a possible complement or alternative to few-shot in-context learning for very large models.Standard finetuning of models larger than 100B can require many accelerators with suitable interconnects.
  • Problems Addressed by Instruction Tuning & Alignment Techniques: Instruction tuning and alignment methods address harmful behavior, non-factual generation, and other deployment challenges in language models.The section frames instruction tuning as part of broader efforts to align models with useful objectives and human preferences.

6 Conclusions

The Flan 2022 collection unifies prior public instruction-tuning resources and adds templates and mixed-prompt improvements. It outperforms several earlier collections across Held-In and Held-Out evaluations and is positioned as a stronger starting point for new tasks.

  • Conclusions: Flan 2022 unifies major public instruction-tuning collections and methods while adding new templates and mixed prompt settings.The collection combines prior resources with simple data and prompting improvements.
  • Conclusions: Flan 2022 outperforms Flan 2021, P3++, Super-Natural Instructions, and OPT-IML-Max 175B across Held-In QA, NLI, Chain-of-Thought, Held-Out MMLU, and BBH tasks.The passage states that these improvements are often large.
  • Conclusions: The collection serves as a more competitive starting point for generalization to new instructions and finetuning on a single new task.This conclusion covers both instruction generalization and applied single-task finetuning.

A Experimental Details

The experiments use the T5X framework and follow the established Flan instruction-tuning setup, while few-shot prompting is generated with varying exemplar counts. Data-source mixture weights are informed by experiments and practitioner intuition.

  • Implementation: The Flan Collection experiments are assembled and run using T5X and follow the setup described in Chung et al. (2022).The passage describes the experimental framework and inherited instruction-tuning setup.
  • Implementation: Few-shot and few-shot Chain-of-Thought prompts are generated with 2, 3, or 5 exemplars during finetuning.The templatizing procedure creates these prompt variants for the experiments.
  • Data balancing: Mixture weights balancing the data sources are informed by Section 3.5 experiments and resulting practitioner intuition.The passage describes how the balancing choices were selected.

A.2 Single-Task Finetuning

The single-task finetuning setup uses a fixed training recipe and selects checkpoints by validation performance. Additional metrics account for label skew in some datasets.

  • Models are finetuned for 100,000 steps using a constant learning rate of 0.001, dropout probability of 0.1, and batches of 128 length-512 sequences.
  • A checkpoint is selected according to the highest validation performance, with held-out training examples supplying validation when needed.For tasks without validation splits, 1,024 training examples are held out; tasks without test splits are handled similarly.
  • Evaluation includes metrics designed to account for label skew in some datasets.

A.3 Evaluation

Evaluation covers held-in QA and NLI tasks, held-out MMLU and BIG-Bench Hard tasks, and chain-of-thought datasets, with scores aggregated across each suite.

  • Held-In: Held-In is the mean accuracy across eight validation tasks: four QA datasets and four NLI datasets.The QA tasks are BoolQ, ARC Easy, ARC Challenge, and AI2’s Middle School Science Exams; the NLI tasks are ANLI R1–R3 and RTE.
  • Evaluation datasets: The evaluation dataset inventory is summarized in Table 3, which identifies datasets used for various finetuning and evaluation experiments.ST-FT denotes Single Task Finetuning.
  • Chain-of-Thought: Chain-of-Thought performance is measured by mean accuracy across GSM8K, StrategyQA, SVAMP, Asdiv, and CommonsenseQA.These datasets use prompts requesting step-by-step explanations in target answers.
  • Held-Out: Held-Out evaluation uses MMLU’s 57 exams and BIG-Bench Hard’s 23 tasks, with MMLU removed from training.The BIG-Bench Hard tasks are those where PaLM performed worse than average human annotators.

B Input Inversion Details

Input inversion expands instruction-tuning tasks by reversing input-output relationships across remaining Flan 2022 datasets. The inversions are generated in task-specific forms and mixed with regular examples at a 30% rate.

  • Existing Flan 2021, P3++, and Super-Natural Instructions tasks already contain some implicit input inversions.
  • The remaining Flan 2022 datasets receive inversions, including Dialog, Program Synthesis, and Chain-of-Thought tasks.
  • Dialog inversions request conversation history, while program-synthesis inversions request the coding question solved by a given program.
  • Chain-of-Thought inversions include every permutation of query, answer, and explanation, with at least one component appearing as output.Figure 7 illustrates these permutations for a zero-shot Chain-of-Thought example and pairs them with instruction templates.
  • Inverted examples comprise 30% of the mixed training data, or three inverted examples for every ten regular examples.The rate approximately mirrors prior work and was chosen for simplicity.
Loading 2301.13688v2…