PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?

Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, Luke Zettlemoyer

arXiv:2202.12837v2cs.CLcs.AI

TL;DR

The paper asks which parts of demonstrations explain in-context learning, since the mechanism behind their performance gains is poorly understood. It tests demonstration variants across language models and tasks, finding that random labels barely reduce performance while label space, input distribution, and format remain important. The results suggest that inference-time adaptation can use task structure beyond the ground-truth input-label mapping.

  • Problem

    The paper addresses limited understanding of how in-context learning works and which demonstration aspects contribute to downstream performance.

  • Method

    The authors experimentally vary demonstration labels and other demonstration components across 12 language models and classification or multi-choice datasets.

  • Results

    Random labels barely hurt performance across tasks and models, while performance gains mainly come from specifying the input space, label space, and sequence format.

  • Takeaways & Limitations

    In-context learning can achieve nearly k-shot performance without labeled data by pairing unlabeled inputs with random labels, indicating that demonstrations provide useful task structure beyond label mappings.

  • Takeaways & Limitations

    Macro-level averages can hide dataset-model differences, including a nearly 14% absolute gold-versus-random-label gap for GPT-J on financial_phrasebank.

Abstract

from arXiv · show

Large language models (LMs) are able to in-context learn -- perform a new task via inference alone by conditioning on a few input-label pairs (demonstrations) and making predictions for new inputs. However, there has been little understanding of how the model learns and which aspects of the demonstrations contribute to end task performance. In this paper, we show that ground truth demonstrations are in fact not required -- randomly replacing labels in the demonstrations barely hurts performance on a range of classification and multi-choce tasks, consistently over 12 different models including GPT-3. Instead, we find that other aspects of the demonstrations are the key drivers of end task performance, including the fact that they provide a few examples of (1) the label space, (2) the distribution of the input text, and (3) the overall format of the sequence. Together, our analysis provides a new way of understanding how and why in-context learning works, while opening up new questions about how much can be learned from large language models through inference alone.

1 Introduction

The paper investigates why demonstrations improve in-context learning and finds that correct input-label mappings matter less than other demonstration properties. Label space, input distribution, and sequence format are key contributors to performance.

  • In-context learning improves downstream performance by conditioning on a few input-label demonstrations, but the mechanisms behind this improvement remain poorly understood.
  • Randomly replacing demonstration labels causes only a marginal performance decline across classification and multi-choice tasks.
  • The weak dependence on correct input-label mappings is consistent across 12 models, including GPT-3.
  • The label space and input-text distribution specified by demonstrations are key drivers of in-context learning, regardless of individual label correctness.
  • The demonstrations’ overall sequence format is also crucial, while meta-training magnifies models’ reliance on simpler demonstration properties.
  • The analysis reframes demonstrations as specifying useful task structure rather than primarily teaching the ground-truth input-label mapping.

2 Related Work

Prior work improves the formulation, selection, and training of in-context learning, while comparatively little work explains why it works. This paper provides an empirical analysis centered on demonstration components and their contribution to performance gains.

  • In-context learning was introduced as an inference-based alternative to fine-tuning for adapting large language models to new tasks.
  • Prior studies explore problem formulation, demonstration-example selection, meta-training objectives, and instruction-following variants.
  • Theoretical work frames in-context learning as Bayesian inference that uses demonstrations to recover latent concepts.
  • Other research links in-context performance to term frequencies in pretraining data, leaving the mechanisms behind gains over zero-shot inference less studied.
  • This paper empirically examines finer-grained demonstration aspects and finds that ground-truth input-label mappings have only a marginal effect.

3 Experimental Setup

The experiments evaluate multiple language models and diverse classification or multi-choice datasets under varied demonstration conditions. The setup compares gold-label, random-label, and no-demonstration configurations to analyze which components matter.

  • The analysis uses 12 decoder-only dense language models ranging from 774M to 175B parameters, including GPT-3 and MetaICL.
  • Figure 3 compares no demonstrations with demonstrations containing gold labels or random labels across classification and multi-choice tasks.
  • The evaluation covers 26 low-resource classification and multi-choice datasets spanning sentiment, paraphrase, inference, hate speech, question answering, and sentence completion.
  • The experiments generally use k = 16 uniformly sampled demonstrations, five random seeds, minimal templates, and macro-averaged classification F1 or multi-choice accuracy.

4 Ground Truth Matters Little

Across classification and multi-choice tasks, replacing correct demonstration labels with random or incorrect labels usually causes only small performance losses. The experiments indicate that demonstrations help through their inputs, label space, and formatting more than through exact input-label mappings.

  • Gold labels vs. random labels: Replacing gold labels with random labels causes only marginal performance losses across nearly all models, with drops of 0–5% absolute.The effect is smaller in multi-choice tasks than in classification tasks.
  • Gold labels vs. random labels: Performance is fairly insensitive to the number of correct demonstration labels, and always-incorrect labels can preserve 92–100% of demonstration improvements.The preserved improvements are reported for MetaICL classification, MetaICL multi-choice, and GPT-J multi-choice; GPT-J classification is a larger-drop exception.
  • Ablations: With varying numbers of examples, demonstrations outperform no demonstrations even at k = 4, while gold-to-random-label performance drops remain 0.8–1.6%.Performance does not increase much once k ≥8 for either gold or random labels.
  • Ablations: The random-label trend also holds with manual templates, although manual templates do not always outperform minimal templates.
  • Ablations: The ablations separate demonstrations into input-label mapping, input distribution, label space, and input-label-pair formatting.

5 Why does In-Context Learning work?

The paper separates demonstrations into four potentially useful aspects and tests them with controlled variants. Input distribution, label space, and especially input-label format contribute to performance, while their effects vary across model types.

  • 5 Why does In-Context Learning work?: The analysis isolates four demonstration aspects: input-label mapping, input-text distribution, label space, and input-label format.Controlled variants quantify each aspect’s impact separately.
  • 5.1 Impact of the distribution of the input text: 3–16% absolute performance drops occur with out-of-distribution inputs for several models, showing that in-distribution demonstration text substantially contributes.Direct GPT-J in multi-choice can perform significantly worse than with no demonstrations, while Direct MetaICL is an exception.
  • 5.2 Impact of the label space: 5–16% absolute gaps in direct models show that conditioning on the label space contributes significantly to performance.Channel models show only a 0–2% absolute drop, or sometimes an increase, when the output space is removed.
  • 5.3 Impact of input-label pairing: Removing input-label format is close to or worse than no demonstrations, indicating that the format is crucial for retaining in-context gains.The authors suggest that input-label sequences trigger the model to mimic the overall format and complete the test example accordingly.
  • 5.3 Impact of input-label pairing: 95% and 82% of in-context-learning improvements can be retained with Direct MetaICL using only inputs or only labels when the format is preserved.Across channel models, random pairings retain 82%, 87%, 86%, and 75% of improvements in the reported classification and multi-choice settings.
  • 5.4 Meta-training: Meta-training with an in-context-learning objective magnifies these effects, encouraging greater reliance on simpler demonstration aspects such as format.The paper hypothesizes that format and generated-text space are easier to exploit than input-label mappings.

6 Discussion & Conclusion

The discussion reframes in-context learning as adapting to demonstration-defined distributions and format rather than necessarily learning a new input-label mapping. This interpretation exposes both substantial zero-shot capacity and limits when the needed correspondence is absent from pretraining.

  • 6 Discussion & Conclusion: The paper concludes that gains mainly come from independently specifying the input space and label space, with up to 95% retained using either when the right format is used.Meta-training magnifies these trends.
  • Does the model learn at test time?: Under a strict definition of learning, the findings suggest models do not learn new tasks at test time because they may ignore demonstrated input-label correspondences.The analysis instead suggests reliance on priors from pretraining.
  • Does the model learn at test time?: Under a broader definition, learning includes adapting to the input and label distributions and demonstration format to improve predictions.The experiments indicate that models use these demonstration aspects to achieve performance gains.
  • Capacity of LMs: In-context learning may fail when the needed input-label correspondence is not already captured in the language model.The paper raises extracting stored mappings, improving the language-model objective, or fine-tuning on labeled data as questions for future work.
  • Connection to instruction-following models: The reported findings are extended as a hypothesis for instruction-following models, where instructions recover existing capacity rather than supervise novel task semantics.The paper notes that this interpretation has only been partially verified and leaves further analysis for future work.
  • Broader implications: Nearly k-shot performance can be achieved without labeled data by pairing unlabeled inputs with random labels, indicating a higher zero-shot baseline than naive zero-shot accuracy suggests.This result is described as significantly improved zero-shot performance.

Limitation

The paper’s evidence is bounded by its task and dataset choices: results focus on established NLP classification and multi-choice benchmarks, and dataset-level behavior varies. Generation remains future work because constructing suitable incorrect outputs while preserving output distributions is nontrivial.

  • The experiments focus on established NLP benchmarks with natural-language inputs, so synthetic tasks may rely more on ground-truth labels.
  • Macro-level averages can conceal dataset-specific gaps, including nearly 14% absolute on financial_phrasebank with GPT-J.
  • The experiments cover classification and multi-choice tasks, while open-set generation is left for future work.
  • Extending the analysis to generation is nontrivial because incorrect input-output correspondences must preserve the correct output distribution.

A Full Datasets

The dataset description reports evaluation using HuggingFace data and development sets, following prior work, with dataset statistics provided in Table 2.

  • The study uses the HuggingFace version of the data.
  • Evaluation uses development data, following Ye et al. (2021).
  • Dataset statistics are reported in Table 2.

B Experimental Details

The experiments use minimal or model-standard demonstration formats, with variants that manipulate label accuracy, input distribution, vocabulary, and other demonstration properties across a 26-dataset benchmark.

  • Example template: Minimal templates concatenate multiple inputs and use each dataset’s label words unchanged; manual templates are not consistently better.
  • Format of the demonstrations: Demonstration formatting separates inputs and labels with spaces or newlines, with model-specific separators between examples.
  • The benchmark contains 26 datasets spanning 6 task categories, with k random training samples used for k-shot evaluation.
  • Details in variants of the demonstrations: For a% accurate demonstrations, k × a/100 pairs are correct and the remaining pairs receive randomly sampled incorrect labels.
  • Details in variants of the demonstrations: Out-of-distribution demonstrations use CC-News sentences sampled to match test-input length, while random-English-word variants draw from a 61,569-word set.
  • Figure 11 evaluates all models on 3 classification and 3 multi-choice datasets to make results comparable across models.

C More Experimental Results

Additional results in Figure 11 use a common interface and evaluate all models on the same six datasets, enabling direct cross-model comparison.

  • Figure 11 shares the interface used by Figure 3.
  • All models are evaluated on 3 classification and 3 multi-choice datasets.
  • The common dataset set makes the models comparable to each other.

C.2 Random labels from true distribution of labels & Task breakdown

Sampling demonstration labels from the training-data label distribution reduces the gold-label performance gap more than uniform random labels. Across most datasets, the gap remains small, although some datasets are notable outliers.

  • 1.3% with Channel MetaICL and 3.5% with Channel GPT-J are the absolute gaps when random labels follow the training-data label distribution.These gaps are smaller than the corresponding uniform-random-label gaps of 1.9% and 5.0%.
  • The gap between gold and random labels is smaller than previously thought across most datasets.Figure 12 reports this trend per dataset for both uniform and training-distribution random labels.
  • Financial_phrasebank and several hate speech detection datasets are outliers with non-negligible performance gaps.The paper leaves investigation of which tasks use correctly paired training data for future work.

C.3 More variants of the demonstrations

The paper tests additional demonstration variants to separate the effects of label space, input distribution, and sequence format. These results reinforce that demonstration format and task-specific conditions matter, while the findings remain bounded by the evaluated task types and datasets.

  • More variants of the demonstrations: Constant labels perform worse than random English labels because they change the demonstration format while removing label-space information.The constant label “answer” can function as a separator between demonstration examples.
  • More variants of the demonstrations: Replacing demonstration inputs with the test input and pairing them with random labels performs significantly worse than most other methods.Although this variant matches the test-input distribution and random labels perform well elsewhere, it does not preserve the same successful behavior.
  • More variants of the demonstrations: Absolute performance differences are similar across model sizes, but relative differences are larger for larger models.The paper attributes this pattern to larger models having higher absolute performance.
  • More variants of the demonstrations: Applying in-context learning to problems whose input-label correspondences were sparse or absent during pretraining may be risky.The authors suggest collecting labeled data and fine-tuning the model in such cases.
  • More variants of the demonstrations: Nearly 14% absolute is the largest reported ground-truth versus random-label gap, occurring on financial_phrasebank with GPT-J.The paper emphasizes that macro-level averages can conceal larger gaps in particular dataset-model pairs.
  • More variants of the demonstrations: The findings are limited to classification and multi-choice tasks, while extending the analysis to generation is nontrivial because randomizing outputs can alter both correspondence and output distribution.The paper also notes that synthetic tasks with limited inputs may use ground-truth labels more, and that different datasets can behave differently.
Loading 2202.12837v2…