PaperlayerView exact revision on arXiv

Source-linked AI summary

Meta-learning via Language Model In-context Tuning

Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, He He

arXiv:2110.07814v2cs.CLcs.LG

TL;DR

Few-shot learning asks how models can adapt to new tasks from very few labeled examples. The paper fine-tunes language models to perform this adaptation through in-context sequence prediction, using instructions, demonstrations, and a target input. In-context tuning outperforms raw prompting and MAML on LAMA and BinaryClfs while reducing sensitivity to demonstration choices and ordering.

  • Problem

    Few-shot learning requires models to adapt to new tasks from very few labeled examples, but raw language-model prompting is sensitive to example ordering, choices, and instruction wording.

  • Method

    In-context tuning fine-tunes a pre-trained language model across tasks to predict a target label from a sequence containing the task instruction, labeled examples, and target input.

  • Results

    In-context tuning improves over raw prompting by 7.6 Precision@1 points on LAMA and 10.6% AUC-ROC score on BinaryClfs, and over MAML by 2.8 and 5.1 points, respectively.

  • Takeaways & Limitations

    In-context tuning directly learns to use few-shot examples and is less sensitive than raw prompting to example ordering, example selection, and instruction wording.

  • Takeaways & Limitations

    The work evaluates only binary classification and knowledge retrieval, at most 5 in-context examples, and models with fewer than 1 billion parameters.

Abstract

from arXiv · show

The goal of meta-learning is to learn to adapt to a new task with only a few labeled examples. To tackle this problem in NLP, we propose $\textit{in-context tuning}$, which recasts adaptation and prediction as a simple sequence prediction problem: to form the input sequence, we concatenate the task instruction, the labeled examples, and the target input to predict; to meta-train the model to learn from in-context examples, we fine-tune a pre-trained language model (LM) to predict the target label from the input sequences on a collection of tasks. We benchmark our method on two collections of text classification tasks: LAMA and BinaryClfs. Compared to first-order MAML which adapts the model with gradient descent, our method better leverages the inductive bias of LMs to perform pattern matching, and outperforms MAML by an absolute $6\%$ AUC ROC score on BinaryClfs, with increasing advantage w.r.t. model size. Compared to non-fine-tuned in-context learning (i.e. prompting a raw LM), in-context tuning directly learns to learn from in-context examples. On BinaryClfs, in-context tuning improves the average AUC-ROC score by an absolute $10\%$, and reduces the variance with respect to example ordering by 6x and example choices by 2x.

1 Introduction

The paper addresses few-shot task adaptation by fine-tuning language models to learn from labeled examples provided in context. In-context tuning improves over raw prompting and MAML while reducing prompting sensitivity.

  • Motivation: Few-shot learning aims to adapt quickly to new tasks using very few labeled examples, unlike systems that often depend on large labeled datasets.The paper frames few-shot learning as important for NLP and contrasts it with conventional data-hungry machine learning.
  • Motivation: Raw language-model prompting casts task learning and prediction as sequence prediction over an instruction, input-output examples, and a target input.For sentiment classification, the model predicts the label by comparing the likelihood of candidate continuations such as “Yes” and “No”.
  • Motivation: Raw LMs are not optimized for in-context few-shot learning and can be highly sensitive to example ordering, example choices, and instruction wording.Recency bias can assign higher probability to labels appearing closer to the target, making accuracy sensitive to demonstration order.
  • Approach: In-context tuning fine-tunes an LM on the sequence-prediction objective across many tasks, then adapts through in-context learning with frozen parameters.Unlike MAML, it avoids bi-level optimization during meta-training while retaining a single model shared across tasks.
  • Results: 7.6 Precision@1 points on LAMA and 10.6% AUC-ROC score on BinaryClfs are the gains over raw LM prompting.Variance also falls for example ordering by 68% on LAMA and 83% on BinaryClfs, and for example choices by 56% and 40%, respectively.
  • Results: 2.8 points on LAMA and 5.1 points on BinaryClfs are the gains over first-order MAML, with increasing advantage as models become larger.The paper attributes this comparison to in-context tuning’s stronger use of language-model inductive bias for extrapolating from examples.

2 Approach

The approach trains a language model to predict task labels from instructions and demonstrations, then performs adaptation by concatenating a new task’s examples with its target input. It is distinguished from gradient-based baselines by frozen parameters during adaptation.

  • 2.1 Problem Setup: The problem setup trains across tasks with natural-language instructions and examples, then evaluates on a new task given only K labeled examples.The target task input is distinct from the model input, which concatenates instructions, demonstrations, and the target.
  • 2.2 In-context Tuning Algorithm: In-context tuning meta-trains a pre-trained LM to predict the target label from a concatenation of the task instruction, labeled examples, and target input.The training objective directly optimizes few-shot in-context learning across a wide variety of tasks.
  • 2.2 In-context Tuning Algorithm: At test time, in-context tuning concatenates new-task examples, the task instruction, and the target input, then predicts without performing a gradient update.Model parameters remain frozen during task adaptation.
  • 2.3 Gradient-based Task Adaptation: MAML adapts by gradient descent on K examples and meta-trains an initialization through inner task optimization and outer optimization of that initialization.This produces a task-specific parameter state from the shared initialization.
  • 2.3 Gradient-based Task Adaptation: MAML’s bi-level optimization has been found to be empirically unstable, hyperparameter-sensitive, and computationally expensive in few-shot settings.Few-shot adaptation is especially sensitive to optimization hyperparameters when large validation sets may be unavailable.
  • 2.2 In-context Tuning Algorithm: In-context tuning removes bi-level optimization by treating adaptation and task-specific prediction as one sequence-prediction problem with task examples in the input.The paper categorizes it as the only approach that directly optimizes the few-shot objective without gradient-based adaptation.
  • 2.4 Other Baselines: Raw in-context learning evaluates an unfine-tuned LM, whereas instruction tuning predicts from the instruction and target input without demonstration examples.These baselines separate the effects of fine-tuning and few-shot examples.

3 Experimental Setup

The experiments evaluate in-context tuning on LAMA and BinaryClfs using task instructions, few-shot examples, and dataset-specific evaluation protocols. They also specify filtering, model architectures, validation, and sampling procedures.

  • Datasets: The evaluation uses LAMA for factual and commonsense knowledge and BinaryClfs for diverse binary classification tasks.LAMA contains 29 relations and 12k examples after filtering; BinaryClfs contains 204 tasks, including 73 testing tasks.
  • Datasets and Metrics: LAMA treats each relation as a task and predicts an object entity from a 21k-word vocabulary, reporting mean P@1 across tasks.The experiments use 8-fold cross-validation because LAMA has no official train-validation-test split.
  • Datasets and Metrics: BinaryClfs evaluates binary classification across tasks described by one to four yes/no questions, using AUC-ROC for each instruction-task pair.Training and testing tasks are grouped by similarity to prevent similar tasks from crossing the split.
  • Dataset Processing: The modified datasets impose scope constraints, excluding LAMA tasks with inputs exceeding 230 BPE tokens and a spam task with a small test set.The reported numbers are not directly comparable to other work because both datasets were modified.
  • Architectures: The study uses BERT-family models for LAMA and GPT2 models for BinaryClfs, spanning model sizes from 110M to 900M parameters.The LAMA models are BERT-Base, BERT-Large, and DeBERTa-XLarge-V2; BinaryClfs uses GPT2-Medium and GPT2-Large.
  • Sampling: Training and evaluation randomly vary task instructions and few-shot examples, while test accuracy averages over sampled instruction-example combinations.The sampling-based average approximates the intractable average over all possible few-shot choices.

4 Results

In-context tuning consistently improves few-shot performance over raw LM prompting, instruction tuning, and MAML while reducing sensitivity to examples and instructions. These gains arise across LAMA and BinaryClfs, with strongest benefits where instructions or raw prompting perform poorly.

  • Few-shot Learning Performance: In-context tuning consistently out-performs raw LM prompting by 7.6 points on LAMA and 10.6 points on BinaryClfs.The averages span model sizes and numbers of few-shot examples.
  • Few-shot Learning Performance: 5-shot in-context tuning out-performs instruction tuning by 6.1 points on LAMA and 4.0 points on BinaryClfs.Performance gains increase as the number of shots increases.
  • Few-shot Learning Performance: 5-shot in-context tuning out-performs 5-shot MAML by 2.8 points on LAMA and 5.1 points on BinaryClfs.The comparison is averaged across model sizes; the authors attribute the advantage to better use of pretrained LMs’ pattern-matching bias.
  • Sensitivity Analysis: In-context tuning remains effective without task instructions and reduces their gain from 5.1 points with raw in-context learning to 1.8 points on BERT-Base and BERT-Large.Without instructions, it out-performs raw in-context learning by an average margin of 9.5 points on LAMA.
  • Sensitivity Analysis: In-context tuning reduces sensitivity to example ordering by 68% on LAMA and 83% on BinaryClfs compared with raw LM prompting.The analysis decomposes accuracy variance across instruction wording, example choices, and example ordering.
  • Sensitivity Analysis: In-context tuning reduces sensitivity to example choices by 56% on LAMA and 40% on BinaryClfs across model sizes.The authors conjecture that exposure to varied example selections and orderings during tuning explains this robustness.
  • Sensitivity Analysis: In-context tuning reduces instruction-wording variance by 19% on LAMA and is less sensitive than raw prompting in five of six evaluation settings.It is especially effective on task instructions with low accuracy under instruction tuning, with a strong negative correlation of -0.910 between instruction-tuning accuracy and gain.

5 Related Work

The paper positions in-context tuning within work on LM prompting, meta-learning, multitask learning, and instruction learning. It distinguishes its approach by directly optimizing few-shot in-context learning and evaluating robustness to prompting artifacts.

  • LM Prompting for FSL: Pretrained LMs can perform few-shot tasks from natural-language instructions and several examples, but prompting is sensitive to instruction wording and example selection and ordering.The paper describes its work as the first to show that an explicit few-shot objective reduces these sensitivities.
  • Meta-learning for FSL: Optimization-based meta-learning adapts task-agnostic models through gradient descent, but few-shot fine-tuning is hyperparameter-sensitive and nested optimization can be unstable.In-context tuning instead uses task examples as part of the input sequence during adaptation.
  • Multi-task Learning: Multitask learning trains one model on the union of multiple task datasets and then fine-tunes it on task-specific examples, whereas meta-learning targets adaptation across tasks.The supplied related-work passage introduces this distinction while noting benefits on downstream tasks with small training sets.
  • Fine-tuned LMs for Instruction Learning: Instruction-learning work fine-tunes LMs across many tasks to leverage their inductive bias, while this paper applies that idea to the more general few-shot meta-learning setting.The paper reports outperforming both instruction tuning and existing few-shot meta-learning methods.
  • Concurrent Work: Compared with concurrent Seq2Seq work, this paper evaluates in-context tuning against MAML and studies prompting oversensitivity; the studies differ in their conclusions about InsT + FT.The paper attributes the discrepancy partly to the concurrent work using 16-shot settings, which may favor gradient-based methods.

6 Future Directions

Future work should extend in-context tuning beyond the paper’s limited task, example, and model settings while addressing distribution shifts and unresolved properties of in-context learning.

  • The work considers only binary classification and knowledge retrieval tasks, at most 5 in-context examples, and models with fewer than 1 billion parameters.
  • The framework could be scaled to broader sequence-to-sequence tasks, more few-shot examples, and larger models.More examples require longer context sizes.
  • The method assumes few-shot training examples and test examples come from the same distribution, which may fail under composition, subgroup, distribution, or adversarial shifts.
  • Important open questions include whether in-context learning is robust to distribution shift and whether it can combine with gradient learning.

7 Conclusion

The paper proposes meta-learning through in-context tuning, which reframes few-shot adaptation and prediction as sequence prediction. It reports stronger accuracy than several baselines and lower sensitivity to prompting choices than raw language models.

  • In-context tuning reframes few-shot task adaptation and prediction as sequence prediction using labeled examples and the target example as input.
  • In-context tuning outperforms raw LM prompting, MAML, and instruction tuning in accuracy across the evaluated benchmarks.
  • In-context tuning is less sensitive than raw LM prompting to few-shot example choices and instruction wording.
  • The authors conjecture that prompting without direct few-shot optimization underestimates large LMs’ few-shot potential and preserves artifacts such as sensitivity to ordering and selection.

A Hyperparameters

The appendix reports the hyperparameters tuned for in-context tuning, MAML, and instruction tuning followed by task fine-tuning on LAMA and BinaryClfs.

  • In-Context Tuning (ours): The experiments tune training epochs and learning rates for in-context tuning separately on LAMA and BinaryClfs.
  • MAML: MAML tuning varies adaptation steps and learning rate, using the same learning rate for inner and outer optimization.
  • Instruction-Tuning + Fine-tuning: Instruction tuning uses the same hyperparameter set as in-context tuning, while downstream task fine-tuning varies epochs and learning rate.
Loading 2110.07814v2…