PaperlayerView exact revision on arXiv

Source-linked AI summary

Larger language models do in-context learning differently

Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, Tengyu Ma

arXiv:2303.03846v2cs.CL

TL;DR

The paper asks how much ICL depends on pretrained semantic priors versus input–label mappings shown in context. It tests flipped-label and semantically-unrelated-label ICL across model families and finds that overriding priors and learning mappings without semantic labels emerge mainly with scale, while instruction tuning favors semantic-prior use more strongly.

  • Problem

    The paper investigates how language models combine pretrained semantic priors with input–label mappings presented through in-context exemplars.

  • Method

    The study compares flipped-label ICL and SUL-ICL across GPT-3, InstructGPT, Codex, PaLM, and Flan-PaLM models with varying scale and instruction tuning.

  • Results

    Overriding semantic priors and performing SUL-ICL emerge primarily with scale; instruction tuning improves mapping learning but strengthens semantic-prior use more.

  • Takeaways & Limitations

    Large language models can learn contradictory or semantically unsupported input–label mappings in context, including linear classification in SUL-ICL settings.

  • Takeaways & Limitations

    QNLI and WSC were excluded because even the largest models performed weakly on them in many settings, and InstructGPT and davinci were omitted from one experiment due to cost.

Abstract

from arXiv · show

We study how in-context learning (ICL) in language models is affected by semantic priors versus input-label mappings. We investigate two setups-ICL with flipped labels and ICL with semantically-unrelated labels-across various model families (GPT-3, InstructGPT, Codex, PaLM, and Flan-PaLM). First, experiments on ICL with flipped labels show that overriding semantic priors is an emergent ability of model scale. While small language models ignore flipped labels presented in-context and thus rely primarily on semantic priors from pretraining, large models can override semantic priors when presented with in-context exemplars that contradict priors, despite the stronger semantic priors that larger models may hold. We next study semantically-unrelated label ICL (SUL-ICL), in which labels are semantically unrelated to their inputs (e.g., foo/bar instead of negative/positive), thereby forcing language models to learn the input-label mappings shown in in-context exemplars in order to perform the task. The ability to do SUL-ICL also emerges primarily with scale, and large-enough language models can even perform linear classification in a SUL-ICL setting. Finally, we evaluate instruction-tuned models and find that instruction tuning strengthens both the use of semantic priors and the capacity to learn input-label mappings, but more of the former.

1 INTRODUCTION

The paper studies how semantic priors and in-context input–label mappings interact in ICL. Across flipped-label and semantically-unrelated-label settings, the ability to rely on mappings rather than priors emerges with scale, while instruction tuning strengthens both tendencies but favors semantic priors.

  • INTRODUCTION: The paper examines whether models perform ICL using semantic prior knowledge, input–label mappings, or both.Regular ICL can support either strategy because natural-language labels provide semantic information while exemplars provide mappings.
  • INTRODUCTION: Flipped-label ICL tests whether models can override semantic priors when exemplar labels contradict them.Evaluation labels remain unchanged, so following flipped exemplars in binary classification should produce accuracy below 50%.
  • INTRODUCTION: SUL-ICL removes semantic information from labels, forcing models to learn input–label mappings from exemplars.The setup replaces task-related labels such as “negative/positive” with unrelated labels such as “foo/bar.”
  • INTRODUCTION: Large models can override semantic priors with flipped labels, whereas small models generally do not.This ability emerges with model scale despite the stronger priors larger models may possess.
  • INTRODUCTION: Learning mappings without semantic label priors also emerges with scale, with some tasks requiring substantial scaling.For example, only PaLM-540B achieves above-random performance on some SUL-ICL datasets.
  • INTRODUCTION: Instruction tuning improves SUL-ICL but makes models more reluctant to follow flipped labels, strengthening semantic-prior use more than mapping learning.The paper therefore finds asymmetric effects of instruction tuning on the two ICL strategies.

2 EXPERIMENTAL SETUP

The experiments span seven NLP tasks and five language-model families with varying sizes and instruction-tuning conditions. They use few-shot prompts, typically with 16 exemplars per class, and sample 100 evaluation examples per dataset.

  • EXPERIMENTAL SETUP: The study evaluates seven widely used NLP tasks, including sentiment, subjectivity, question, duplicate-question, and entailment classification.The supplied passage identifies the task suite as drawn from established NLP benchmarks.
  • MODELS: The models cover GPT-3, InstructGPT, Codex, PaLM, and Flan-PaLM families.PaLM is evaluated at 8B, 62B, and 540B parameters, while instruction-tuned variants are also included.
  • EXPERIMENTAL DETAILS: Each inference example uses a different set of few-shot exemplars, with k = 16 exemplars per class by default.The study also varies the number of exemplars in additional experiments.
  • EXPERIMENTAL DETAILS: Two additional tasks, QNLI and WSC, are excluded because even the largest models perform weakly in many settings.This exclusion is reported as a preliminary-experiment limitation.

3 INPUT–LABEL MAPPINGS OVERRIDE SEMANTIC PRIORS IN LARGE MODELS

Flipped-label ICL tests whether models follow contradictory in-context mappings instead of semantic priors. This capacity is largely absent in smaller models but emerges in larger models as label flips increase.

  • EXPERIMENT: The experiment progressively flips the labels of in-context exemplars while leaving evaluation labels unchanged.For binary classification, 50% flipped labels correspond to random labels, and perfect mapping-following at 100% flips would yield 0% accuracy.
  • RESULTS: At 100% flipped labels, text-davinci-002 accuracy drops from 90.3% to 22.5%.This well-below-random performance indicates that the model follows flipped exemplars rather than relying only on semantic priors.
  • RESULTS: Small models remain relatively flat and often do not fall below random guessing as exemplar labels are increasingly flipped.GPT-3 models can remove semantic priors to reach guessing accuracy but do not reliably override them.
  • RESULTS: Large models show performance drops well below random guessing as flipped labels increase, demonstrating that they can override pretrained semantic priors.The trend is reported across model families and tasks.
  • CONCLUSION: The ability to override semantic priors with in-context mappings emerges with model scaling.The conclusion follows from the contrasting behavior of small and large models under flipped-label ICL.

4 IN-CONTEXT LEARNING WITH SEMANTICALLY UNRELATED LABELS

SUL-ICL removes semantic information from labels, requiring models to learn input–label mappings from exemplars. This ability improves with scale and emerges sharply for some tasks.

  • Setup: SUL-ICL replaces natural-language targets with semantically unrelated labels such as “Foo” and “Bar,” removing semantic priors from the labels.The setup relabels examples while preserving the underlying task, so successful performance requires learning the mappings shown in-context.
  • Scaling effects: Small models suffer larger performance drops than large models when semantic priors are removed, indicating greater reliance on target meaning.Large models experience very small drops from regular ICL to SUL-ICL and can learn input–label mappings in-context.
  • Scaling effects: Larger models gain more from additional SUL-ICL exemplars than smaller models across the tested model families.The results indicate that larger models make better use of additional input–label mappings presented in context.
  • Emergent tasks: Some SUL-ICL tasks emerge only at sufficient scale: RTE reaches well above random for PaLM-540B, while RTE and ETHOS exceed 80% for code-davinci-002.ETHOS performance spikes earlier for PaLM, from PaLM-8B to PaLM-62B; other datasets can exceed random with smaller models.

5 INSTRUCTION TUNING WITH EXEMPLARS IMPROVES INPUT–LABEL

The paper compares pretraining-only and instruction-tuned models to separate learning input–label mappings from using semantic priors. Instruction tuning improves mapping learning but makes models less willing to override semantic priors.

  • Experimental comparison: Instruction tuning is evaluated by comparing PaLM models with their instruction-tuned Flan-PaLM versions using regular and SUL-ICL experiments.The comparison focuses on PaLM and Flan-PaLM because the base-model relationship between GPT-3 and InstructGPT cannot be established.
  • Input–label mappings: Flan-PaLM performs better than PaLM in SUL-ICL, with Flan-PaLM-8B outperforming PaLM-8B by 9.6%.The improvement is most prominent in smaller models and indicates stronger learning of input–label mappings.
  • Semantic priors: Instruction-tuned models are worse at following flipped labels: Flan-PaLM remains no better than random at overriding semantics even with 100% flipped labels.PaLM reaches as low as 31% accuracy under 100% flipped labels, whereas Flan-PaLM does not override its semantic targets beyond random performance.
  • Overall effect: Instruction tuning improves input–label mapping while concurrently strengthening reliance on semantic priors.The paper concludes that the semantic-prior effect is stronger than the mapping-learning improvement in this comparison.

6 LARGE LANGUAGE MODELS CAN PERFORM LINEAR CLASSIFICATION

The paper tests whether scaled language models can learn high-dimensional classification rules from in-context examples without being given the equation or threshold. Large-enough Codex models outperform random guessing on 16-dimensional classification.

  • Task design: The experiment constructs N-dimensional datasets with exemplars above and below an unknown threshold, requiring classification of new points in SUL-ICL.The model is not told either the equation or the threshold and must infer the relationship from examples.
  • Results: 16-dimensional linear classification emerges with Codex model scale.Accuracy is measured over 100 evaluation examples using k = 16 exemplars per class.
  • Results: The largest Codex model outperforms random guessing by 19%, while smaller models exceed random by no more than 9%.The comparison supports a scaling factor for performing high-dimensional linear classification.

7 RELATED WORK

Prior work has emphasized semantic priors in ICL while also showing that models can learn input–label mappings, especially in synthetic settings. This paper connects those findings to model scaling and qualitatively different behavior.

  • Earlier studies found that random exemplar labels can preserve performance, suggesting ICL often relies on label-space and input-distribution cues rather than ground-truth mappings.
  • Other work indicates that transformers can learn input–label mappings, including through algorithms resembling least-squares estimation, ridge regression, and gradient descent.
  • Scaling studies have shown that larger language models can develop qualitatively different capabilities, including difficult reasoning behaviors.

8 CONCLUSIONS

The paper compares reliance on pretrained semantic priors with learning input–label mappings from context. It finds that scale enables overriding priors, SUL-ICL, and high-dimensional linear classification, while instruction tuning strengthens both tendencies.

  • Large language models can override semantic priors when enough flipped labels contradict prior knowledge, and this ability emerges with model scale.
  • SUL-ICL removes semantic meaning from labels, and successful performance in this setting emerges with model scale.
  • High-dimensional linear classification also emerges with model scale in the paper’s experiments.
  • Instruction tuning improves input–label mapping capacity while also strengthening semantic priors.

A DATASET CREATION

The experiments use natural-language datasets with randomly selected training exemplars and validation examples, alongside specified dataset subsets and label choices.

  • For natural-language tasks, exemplars are randomly selected from training data and evaluation examples from validation data, following prior work.
  • The FP experiments use the sentences_allagree subset, ETHOS uses its binary subset, and TREC uses six coarse labels.

B.1 SUL-ICL IS EASIER THAN FLIPPED-LABEL ICL

The paper compares SUL-ICL with flipped-label ICL using matched model outputs and evaluation procedures. SUL-ICL generally performs better, especially for medium-sized models, supporting the view that overriding semantic priors is harder than learning arbitrary mappings.

  • SUL-ICL is expected to be easier than flipped-label ICL because it does not require overriding contradictory semantic priors.
  • The comparison uses SUL-ICL results from Figure 3 and flipped-label outputs with 100% flipped labels, with evaluation examples also flipped for comparability.
  • SUL-ICL performance is almost always higher than flipped-label performance across the compared models.
  • 74% was the largest reported performance difference, occurring for text-curie-001 on SST-2.
  • Medium-sized models show the largest gap, while small and large models show smaller but still significant differences.
  • Accuracy comparisons are not always exactly complementary because small models sometimes output labels outside the provided label set.

B.2 REMAPPING INPUTS HURTS PERFORMANCE

Remapping input words progressively removes semantic information from SUL-ICL inputs, degrading performance especially for small models. At complete remapping, even the largest models generally cannot outperform random guessing, although TREC is an exception.

  • The remapping procedure replaces a randomly selected proportion of prompt words with other randomly selected words, reducing input semantic meaningfulness.0% remapping leaves inputs unchanged, while 100% remapping completely scrambles the input words.
  • Small models generally approach guessing performance after 25%–50% of input words are remapped.Large models show linear performance drops and usually reach guessing accuracy only at 75%–100% remapping.
  • Larger models are more robust to input noise but cannot consistently unscramble words when enough input words are remapped.Reliable performance at complete remapping would require recovering most mapped words, which is difficult even for humans given the number of words per prompt.
  • TREC is an exception: large models can outperform random guessing despite completely scrambled inputs, though the reason is unclear.

B.3 MANY TARGET TYPES WORK

Large models can learn input–label mappings in SUL-ICL across many target formats and simple prompt templates, while small models are more disrupted by semantically relevant prompts. Some tasks remain too difficult regardless of target type.

  • Many target types work: Large models maintain relatively consistent SUL-ICL performance across numerical, alphabetical, fruit, and other semantically unrelated targets.Reversing numerical targets does not consistently hurt performance as much as flipped-label ICL.
  • Task difficulty: QNLI and WSC do not significantly outperform random guessing for any tested target type.The authors therefore remove these datasets from other experiments.
  • Prompt formats: The experiments compare default Input/Output prompts with Input→Output, paired, Question/Answer, Student/Teacher, and Q/A formats.
  • Prompt formats: SUL-ICL can work with simple formats that present inputs and labels, with mean accuracies ranging from 77.9% to 87.7% across tested formats.The overall mean was 84.2%, and no tested format was significantly better than another.
  • Semantically relevant prompts: The smallest Codex model loses performance across all tested datasets with semantically relevant prompts, while the largest Codex model is relatively unaffected.The middle Codex model shows dataset-dependent changes.

B.6 LARGE MODELS ARE ROBUST TO OUT-OF-DISTRIBUTION DATASETS

Model scale improves robustness in SUL-ICL when datasets or labels differ from familiar conditions. Large models generally lose less performance than small models, though robustness varies across model families and tasks.

  • Out-of-distribution datasets: Small models suffer performance drops of up to 36% on out-of-distribution datasets, while text-curie-001 drops 4% and text-davinci-001 shows no significant change.
  • Flipped-label robustness: At 0% flipped labels, larger models generally perform better, but increasing label flips causes performance drops for large models rather than small models.This trend is especially pronounced for InstructGPT and is also observed in Codex and PaLM.
  • Flipped-label robustness: The SUBJ dataset shows performance drops across all model families and model sizes when labels are flipped.The paper notes that it is unclear why this task more strongly encourages following flipped labels.
  • SUL-ICL robustness: For InstructGPT, Codex, and PaLM, large models generally lose less accuracy than small models when switching from natural-language to semantically unrelated targets.This pattern implies greater capacity to learn input–label mappings when semantic priors are unavailable.
  • Using exemplars: Larger models benefit more than smaller models from additional in-context exemplars in SUL-ICL.

C.3 INSTRUCTION TUNING

Instruction tuning improves learning of input–label mappings in SUL-ICL, but makes models less willing to override semantic priors with flipped labels. The appendix also illustrates the prompt formats used for these settings and the linear-classification evaluation.

  • C.3 INSTRUCTION TUNING: For SST-2, QQP, RTE, and ETHOS, Flan-PaLM models outperform their corresponding PaLM models in SUL-ICL.On SST-2, Flan-PaLM-8B exceeds PaLM-8B by 28% and PaLM-62B by 2%, while SUBJ is an exception where PaLM-8B leads Flan-PaLM-8B by 23%.
  • C.3 INSTRUCTION TUNING: Instruction-tuned models generally learn input–label mappings better than pretraining-only models in SUL-ICL, although some datasets show reduced performance.The per-dataset comparison reports improvements on several tasks but also identifies exceptions such as SUBJ.
  • C.3 INSTRUCTION TUNING: With one exemplar per class, Flan-PaLM often scores near 0% accuracy, whereas this issue generally does not persist with two or more exemplars per class.The reason is unknown and may involve responses falling outside the target response set.
  • C.3 INSTRUCTION TUNING: Across datasets, Flan-PaLM performs better than its corresponding PaLM model in the flipped-label setting.The flipped-label experiment excludes RTE because its prompts consistently exceed the context length.
  • C.3 INSTRUCTION TUNING: PaLM models learn flipped input–label mappings more effectively than Flan-PaLM models, indicating that instruction tuning strengthens semantic priors.PaLM accuracy decreases more as labels are flipped, consistent with greater willingness to follow the flipped mappings.
  • C.4 LINEAR CLASSIFICATION: These experiments generate random N-dimensional vectors and evaluate models with k = 16 in-context exemplars per class against a polynomial SVM.The linear-classification prompts use SUL-ICL targets, with negative examples labeled “Foo” and positive examples labeled “Bar.”
  • C.4 LINEAR CLASSIFICATION: The largest Codex model performs linear classification up to 64 dimensions, whereas smaller Codex models reach guessing performance at approximately 16 dimensions.All PaLM models perform linear classification up to at least 8 dimensions, with little apparent dependence on scale; neither family outperforms the SVM baseline.
  • D FULL PROMPT EXAMPLES: The appendix provides full few-shot prompts for seven datasets and explains that SUL-ICL or flipped-label prompts are created by swapping the target labels.The default examples use k = 16 exemplars per class and natural-language targets for regular ICL.
Loading 2303.03846v2…