PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?

Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, Luke Zettlemoyer

arXiv:2202.12837v2cs.CLcs.AI

TL;DR

Despite limited understanding of what makes demonstrations effective for in-context learning, this paper examines how their features contribute to task performance. It finds that randomly replacing demonstration labels barely reduces performance, while other aspects of the demonstrations matter.

  • Problem

    It remains poorly understood how in-context learning works and which aspects of demonstrations contribute to end-task performance.

    Source: §1 · p. 1

  • Method

    The paper analyzes demonstrations by varying label correctness and examining how their input space, label space, and sequence format relate to performance.

    Source: §4.2 · p. 4, §6 · p. 8

  • Results

    Randomly replacing demonstration labels barely hurts performance across classification and multi-choice tasks, consistently across 12 models.

    Source: §1 · p. 1

  • Takeaways & Limitations

    The findings indicate that demonstrations can support performance through their input and label spaces and sequence format, even without ground-truth input-label mappings.

    Source: §1 · p. 1, §6 · p. 8

    The analysis reports macro-level averages across NLP datasets, which can behave differently, with some dataset-model pairs showing larger gaps between gold and random labels.

    Source: Limitation · p. 9

Abstract

from arXiv · show

Large language models (LMs) are able to in-context learn -- perform a new task via inference alone by conditioning on a few input-label pairs (demonstrations) and making predictions for new inputs. However, there has been little understanding of how the model learns and which aspects of the demonstrations contribute to end task performance. In this paper, we show that ground truth demonstrations are in fact not required -- randomly replacing labels in the demonstrations barely hurts performance on a range of classification and multi-choce tasks, consistently over 12 different models including GPT-3. Instead, we find that other aspects of the demonstrations are the key drivers of end task performance, including the fact that they provide a few examples of (1) the label space, (2) the distribution of the input text, and (3) the overall format of the sequence. Together, our analysis provides a new way of understanding how and why in-context learning works, while opening up new questions about how much can be learned from large language models through inference alone.

1 Introduction

Although in-context learning outperforms zero-shot inference across many tasks, the paper finds that correct demonstration labels are not necessary for effective performance. Demonstrations still help by conveying the label space, input-text distribution, and sequence format.

1 Introduction

  • Randomizing demonstration labels barely hurts classification and multi-choice performance, a result consistent across 12 models including GPT-3.The result suggests models do not rely on the demonstrations’ input-label mappings to perform the task.Source: §1 · p. 1, Figure 1 · p. 1
  • The label space and input-text distribution in demonstrations both support in-context learning, regardless of whether individual labels are correct.The paper evaluates variants of demonstrations to measure the impact of these aspects.Source: §1 · pp. 1–2
  • When the label space is unknown, random English words as labels work significantly better than providing no labels, showing that overall format is also crucial.Source: §1 · p. 1
  • Meta-training with an in-context learning objective magnifies these effects, with models almost exclusively exploiting simpler demonstration aspects such as format rather than input-label mappings.Source: §1 · p. 2

2 Related Work

Related work presents in-context learning as inference-time task adaptation and notes that comparatively little work explains why it works; this paper positions itself as an empirical analysis of demonstrations’ contribution.

2 Related Work

  • In-context learning adapts to a new task through inference alone, conditioning on training examples as demonstrations without gradient updates.Figure 2 depicts demonstrations as k input-label pairs from training data.Source: §2 · p. 2, Figure 2 · p. 2
  • Prior work improves in-context learning formulations and example selection, explores meta-training and instruction-following variants, and reports brittleness and oversensitivity.Examples include Zhao et al. (2021), Min et al. (2021a), and Wei et al. (2022a); brittleness is reported by Lu et al. (2021), Zhao et al. (2021), and Mishra et al. (2021a).Source: §2 · p. 2
  • Explanations remain less explored: Xie et al. (2022) formalize in-context learning as Bayesian inference using demonstrations to recover latent concepts, while Razeghi et al. (2022) link performance to pretraining term frequencies.The paper contrasts these studies with its empirical analysis of demonstration components.Source: §2 · p. 2
  • The paper claims to provide the first empirical analysis of why in-context learning outperforms zero-shot inference, finding that ground-truth demonstration mappings have only a marginal effect.It measures the impact of finer-grained aspects of demonstrations.Source: §2 · p. 2

3 Experimental Setup

The experiments evaluate in-context learning across a diverse set of models and low-resource classification and multi-choice datasets, using repeated demonstrations and task-appropriate metrics. The supplied figure reports that random-label demonstrations perform very close to gold-label demonstrations.

Models

  • The study evaluates 12 models, including decoder-only dense LMs from 774M to 175B parameters, two inference methods, and MetaICL on datasets excluded from its meta-training.The LM inference methods are direct and channel, following Min et al. (2021a); the set includes GPT-3, fairseq 13B, and MetaICL initialized from GPT-2 Large.Source: §3 · pp. 2–3

Results shown in Figure 3

  • Random-label demonstrations perform very close to gold-label demonstrations across the classification and multi-choice tasks shown.The figure covers 16 classification and 10 multi-choice datasets for the first eight models, and 3 of each for the last four; it points to Figure 11 for numbers comparable across all models.Source: Figure 3 · p. 3

Evaluation Data

  • Evaluation spans 26 low-resource classification and multi-choice datasets across tasks including sentiment, paraphrase detection, natural language inference, hate speech, question answering, and sentence completion.Datasets have fewer than 10K training examples, include GLUE and SuperGLUE benchmarks, and cover domains such as science, social media, and finance.Source: §3 · p. 3

Other Details

  • By default, experiments use 16 uniformly sampled training examples as demonstrations, with five seeds and Macro-F1 for classification or Accuracy for multi-choice tasks.Results are averaged over seeds per dataset and then macro-averaged across datasets; fairseq 13B and GPT-3 use six datasets and three seeds because of resource limits, and inputs use minimal templates.Source: §3 · p. 3

4 Ground Truth Matters Little

Across the tested models, replacing gold demonstration labels with random labels generally causes only small performance losses, and performance remains fairly insensitive to how many labels are correct. The results suggest demonstrations help through more than their input-label pairings, and this pattern persists across different example counts and manual templates.

4.1 Gold labels vs. random labels

  • Replacing gold labels with random labels lowers performance by 0–5% absolute across nearly all models, with average drops of 2.6% in classification and 1.7% in multi-choice.The authors conclude that correct input-label pairs are not necessary for demonstration gains; models appear to recover the task’s expected correspondence from other aspects of the demonstrations.Source: §4.1 · p. 4

4.2 Ablations

  • With always-incorrect labels, demonstrations still outperform no demonstrations, retaining 92% of MetaICL’s classification gain, 100% of its multi-choice gain, and 97% of GPT-J’s multi-choice gain.GPT-J in classification drops nearly 10% with always-incorrect labels, but still significantly beats no demonstrations.Source: §4.2 · pp. 4–5, Figure 4 · p. 4
  • With k = 4, demonstrations significantly outperform no demonstrations; gold-to-random-label drops are usually 0.8–1.6%, and performance grows little beyond k ≥ 8.The k = 4 classification drop has an exception of 4.4%, likely due to high variance; the authors hypothesize that small demonstrations suffice to convey other useful components of the data.Source: Figure 5 · p. 4, §4.2 · p. 5
  • The small performance loss from replacing gold labels with random labels also holds with manual, dataset-specific templates, which do not always outperform minimal templates.The random-label comparison is consistent across most datasets, though the gap can depend on the dataset; sampling labels from the true label distribution narrows it further.Source: §4.1 · p. 4, §4.2 · p. 5, Figure 6 · p. 5

5 Why does In-Context Learning work?

The section examines four possible signals in demonstrations—input-label mapping, input-text distribution, label space, and format—and finds that text distribution, label space, and especially input-label pairing can contribute to performance beyond correct labels.

5.1 Impact of the distribution of the input text

  • OOD inputs lowered performance by 3–16% absolute for Channel MetaICL, Direct GPT-J, and Channel GPT-J across classification and multi-choice; Direct GPT-J fell below no demonstrations on multi-choice.Direct MetaICL was an exception. The authors suggest in-distribution text helps because it makes the task closer to language modeling, which uses in-distribution text during training.Source: §5.1 · p. 6, Figure 8 · p. 6

5.2 Impact of the label space

  • Direct models lost 5–16% absolute with random English-word labels versus labels from the task label space, while channel models changed by 0–2% or sometimes improved.The authors hypothesize that multi-choice tasks also have characteristic choice distributions that models can use, even without a fixed label set.Source: §5.2 · p. 6, Figure 9 · p. 7

5.3 Impact of input-label pairing

  • Removing input-label pairing made performance close to or worse than using no demonstrations, indicating that the sequence format helps models complete a test example as expected.The authors suggest that a sequence of input-label pairs prompts the model to mimic that format when given a new input.Source: §5.3 · p. 7
  • Direct MetaICL retained 95% (classification) and 82% (multi-choice) of gains; channel MetaICL/GPT-J retained 82%/87% (classification) and 86%/75% (multi-choice).These results came from preserving input-label format while using random sentences with the label set for Direct MetaICL, or training inputs paired with random English words for channel models; removing the format was close to or worse than no demonstrations.Source: §5.3 · p. 7, Figure 10 · p. 7

5.4 Impact of meta-training

  • MetaICL showed nearly zero influence from input-label mapping and input distribution in Direct mode, and from input-label mapping and output space in Channel mode.The authors hypothesize that meta-training encourages models to exploit simpler demonstration features, such as format, and ignore harder ones, such as the input-label mapping.Source: §5.4 · p. 8

6 Discussion & Conclusion

Ground-truth demonstration labels matter less than expected: performance gains mainly come from specifying the input and label spaces and using the right format. The discussion distinguishes learning a new input-label mapping from adapting to demonstration-provided distributions and format, and identifies limits and future directions for this account.

Discussion & Conclusion

  • Replacing gold demonstration labels with random labels only marginally lowers performance; either inputs alone or the label set alone can retain up to 95% of gains in the right format.The paper attributes gains mainly to independently specifying the input and label spaces.Source: §6 · p. 8
  • The direct model exploits the label space better than the input distribution, whereas the channel model exploits the input distribution better than the label space.Source: §6 · p. 8
  • Under a strict definition of learning as capturing the demonstrated input-label correspondence, LMs do not learn new tasks at test time; under a broader definition, they adapt to input and label distributions and format.The broader account includes using demonstration aspects to improve prediction accuracy.Source: §6 · p. 8
  • The findings suggest that pretraining supplies input-label correspondences, but in-context learning may fail when the needed correspondence is absent from the model.Possible routes forward include better extracting stored mappings, broadening task semantics through the LM objective, or fine-tuning on labeled data.Source: §6 · p. 8
  • The authors hypothesize that instructions, like demonstrations, prompt models to recover existing capacity rather than teach novel task semantics, and leave analysis of instruction-following models for future work.Source: §6 · pp. 8–9
  • Nearly k-shot performance is possible without labeled data by pairing each unlabeled input with a random label as a demonstration.This finding implies the zero-shot baseline is higher than previously thought.Source: §6 · p. 9
  • Future work could improve zero-shot performance under relaxed assumptions about access to unlabeled training data.Source: §6 · p. 9

Limitation

The paper’s analysis is limited to established NLP benchmarks with natural-language inputs, and its macro-level findings mask variation across datasets. Generation remains untested, while extending the analysis to it poses a challenge: varying input-output correspondence without changing the output distribution.

Limitation

  • The study focuses on established NLP benchmarks with natural-language inputs; synthetic tasks with more limited inputs may rely more on ground-truth labels, as observed by Rong (2021).Different dataset-model pairs can behave differently, so the paper’s macro-level analysis may obscure variation.Source: Limitation · p. 9
  • Nearly 14% absolute was the most extreme gap between ground-truth and random labels, on financial_phrasebank with GPT-J, despite the paper reporting averages across datasets.Kim et al. (2022) also found that negated labels substantially lower classification performance, but estimated random-label performance by interpolation; this paper samples labels uniformly.Source: Limitation · p. 9
  • The experiments cover only classification and multi-choice tasks, leaving open whether ground-truth outputs are needed for open-set tasks such as generation.Testing this is nontrivial: demonstrations must have incorrect input-output correspondence while preserving the correct output distribution.Source: Limitation · p. 9
  • In chain-of-thought prompting, randomly pairing rationales across examples significantly degrades performance, while other counterfactual rationales, such as wrong equations, degrade it less than expected (Madaan and Yazdanbakhsh, 2022).The paper refers readers to Madaan and Yazdanbakhsh (2022) for further discussion of which rationale aspects matter.Source: Limitation · p. 9

A Full Datasets

The appendix uses 26 low-resource datasets, with the same k-shot training data across five random seeds and development data for evaluation.

A Full Datasets

  • The study includes 26 datasets, following the low-resource dataset selection in Min et al. (2021b).The set spans sentiment and paraphrase tasks, hate-speech and stance classification, and multiple-choice question answering and reasoning datasets.Source: A Full Datasets · p. 13
  • The experiments use the HuggingFace data version, the same k-shot training examples across five random seeds, and development data for evaluation.Development-set evaluation follows Ye et al. (2021).Source: A Full Datasets · p. 13

B Experimental Details

Experiments primarily use minimal input templates and each model’s standard demonstration formatting. The section also specifies the dataset set and how altered demonstrations are constructed.

Example template

  • Main experiments use minimal templates with dataset label words unchanged; manual templates from prior work are explored but are not consistently better.Example templates are listed in Table 3.Source: B Experimental Details · p. 13

Format of the demonstrations

  • Demonstration formatting follows each model’s standard: GPT-2 uses spaces between examples, MetaICL, GPT-J, and GPT-3 use three newlines, and fairseq uses newlines.Inputs and labels are separated by a space for GPT-2 and by a newline for the other listed models.Source: B Experimental Details · p. 13

Dataset selection

  • The experiments use 26 datasets across six task categories, with k random training examples used for k-shot evaluation.Table 2 reports each dataset’s train and test example counts; its training counts refer to the original training datasets.Source: Table 2 · p. 13

Details in variants of the demonstrations

  • For a% label accuracy, demonstrations contain k × a/100 correct pairs and k × (1 − a/100) incorrect pairs in random order.Incorrect labels are sampled from the label set excluding the gold label.Source: B Experimental Details · pp. 13–14
  • OOD demonstrations are sampled from CC-News with sentence lengths matched to test inputs; random-English-word demonstrations draw from a 61,569-word list.The word list is from pypi.org/project/english-words.Source: B Experimental Details · pp. 13–14

C More Experimental Results · C.1 Gold labels vs. random labels

Figure 11 evaluates all models on the same three classification and three multi-choice datasets, making their results comparable.

C.1 Gold labels vs. random labels

  • Figure 11 evaluates all models on three classification and three multi-choice datasets, making their results comparable.It uses the same interface as Figure 3.Source: §C.1 · p. 14

C.2 Random labels from true distribution of labels & Task breakdown

Sampling demonstration labels from the training data’s true label distribution reduces the gold-label performance gap compared with uniformly random labels. This smaller gap holds across most datasets, though some are outliers.

C.2 Random labels from true distribution of labels & Task breakdown

  • For classification tasks, this variant samples demonstration labels from the training data’s true label distribution rather than uniformly from the label space.The uniform-label variant was used in Section 4.Source: §C.2 · p. 14
  • The gold-label gap falls from 1.9% to 1.3% absolute with Channel MetaICL and from 5.0% to 3.5% absolute with Channel GPT-J when random labels follow the training distribution.These gaps are smaller than when labels are sampled uniformly from the label space.Source: §C.2 · p. 14
  • The smaller gap between gold and random labels is consistent across most datasets, with financial_phrasebank and some hate speech datasets as outliers.The paper suggests future work investigate which tasks rely more on correctly paired training data.Source: §C.2 · p. 14

C.3 More variants of the demonstrations

This section examines how alternative demonstration constructions affect classification and multi-choice performance, including variants that change sequence format. It also notes that extending the analysis to open-set tasks such as generation is nontrivial because preserving the correct output distribution while removing input-output correspondence is unclear.

C.3 More variants of the demonstrations

  • Constant-label demonstrations performed consistently worse than demonstrations with random English labels, which the authors attribute to constant labels changing the format by acting as separators between examples.This variant replaces all demonstration labels with the constant text “answer,” removing the label-space effect while retaining the input-text distribution.Source: §C.3 · p. 14
  • Both variants suggest that preserving the input distribution or using random labels alone is insufficient when the demonstration sequence format changes.The proposed explanation in each case is that the constant label or repeated input functions as a separator between examples.Source: §C.3 · pp. 14–15
  • Repeating the test input in every demonstration with random labels performed significantly worse than most other methods, despite matching the test input distribution and using random labels.The authors suggest that the repeated input changes sequence format because it can act as a separator between examples.Source: §C.3 · pp. 14–15
  • The analysis covers classification and multi-choice tasks; extending it to open-set tasks such as generation is nontrivial because removing input-output correspondence while preserving the correct output distribution is unclear.Source: §C.3 · p. 19

Related papers

Loading 2202.12837v2…