Source-linked AI summary
What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning
Jane Pan, Tianyu Gao, Howard Chen, Danqi Chen
TL;DR
The paper asks whether ICL recognizes pre-trained tasks or learns new mappings from demonstrations. It disentangles task recognition and task learning with controlled experiments across classification datasets and three LLM families, finding that recognition does not scale while learning emerges with scale and more demonstrations.
Problem
There is no consensus on whether ICL mainly applies pre-trained task knowledge or performs implicit learning over demonstrations.
Method
The paper separates recognition from learning using RANDOM labels for recognition and ABSTRACT labels and minimal prompts for new-mapping learning, across GPT-3, LLaMA, and OPT.
Results
Task recognition achieves non-trivial performance but remains largely flat with model size and demonstrations, whereas task learning improves with both.
Takeaways & Limitations
ICL comprises distinct task-recognition and task-learning phenomena that should be distinguished when interpreting results.
Takeaways & Limitations
The study is limited to classification tasks, and the mechanisms underlying task learning remain elusive.
Abstract
from arXiv · showhide
Large language models (LLMs) exploit in-context learning (ICL) to solve tasks with only a few demonstrations, but its mechanisms are not yet well-understood. Some works suggest that LLMs only recall already learned concepts from pre-training, while others hint that ICL performs implicit learning over demonstrations. We characterize two ways through which ICL leverages demonstrations. Task recognition (TR) captures the extent to which LLMs can recognize a task through demonstrations -- even without ground-truth labels -- and apply their pre-trained priors, whereas task learning (TL) is the ability to capture new input-label mappings unseen in pre-training. Using a wide range of classification datasets and three LLM families (GPT-3, LLaMA and OPT), we design controlled experiments to disentangle the roles of TR and TL in ICL. We show that (1) models can achieve non-trivial performance with only TR, and TR does not further improve with larger models or more demonstrations; (2) LLMs acquire TL as the model scales, and TL's performance consistently improves with more demonstrations in context. Our findings unravel two different forces behind ICL and we advocate for discriminating them in future ICL research due to their distinct nature.
1 Introduction
The paper addresses disagreement over whether in-context learning primarily recognizes pre-trained tasks or learns new mappings from demonstrations. It separates task recognition and task learning and reports that they emerge under different conditions.
- ICL mechanisms lack consensus, with prior work emphasizing either pre-trained task recognition or implicit learning from demonstrations.
- The paper disentangles task recognition, which applies pre-trained priors, from task learning, which acquires new input-label mappings.
- RANDOM restricts models to task recognition, while ABSTRACT tests task learning with labels replaced by abstract symbols unseen with the inputs during pre-training.
- Experiments span classification datasets and GPT-3, LLaMA, and OPT, varying model sizes and demonstration counts.
- Task recognition remains non-trivial but does not scale with model size or demonstrations, whereas task learning emerges with larger models and more examples.
- The findings provide a framework for understanding ICL behavior by distinguishing two forces with different scaling patterns.
2 Task Recognition and Task Learning
The paper defines task recognition and task learning as distinct ways models use demonstrations. Controlled label manipulations isolate recognition from learning and examine how each responds to scale.
- Task recognition identifies a mapping from input distributions and applies pre-trained priors without relying on input-label pairs.
- Task learning captures a new input-label mapping from demonstrations, unlike recognition of an already learned task.
- Recognition appears at small scale without strong improvement from size or demonstrations, while learning improves as both increase.
- GOLD provides natural prompts and gold labels, jointly reflecting task recognition and task learning.
- RANDOM uses uniformly sampled labels with natural prompts to restrict models to task recognition.
- ABSTRACT uses minimal prompts and semantically unclear labels, with randomized mappings evaluated against remapped targets to reflect task learning.
OPT LLaMA GPT-3
The evaluation averages accuracy across 16 datasets and compares GPT-3, LLaMA, and OPT across model sizes and demonstration counts.
- Figure 2 averages accuracy across 16 datasets for GPT-3, LLaMA, and OPT.
- Its top graphs compare model sizes using 32 examples, while its bottom graphs compare demonstration counts for davinci, LLaMA-65B, and OPT-66B.
- The experiments use three settings—GOLD, RANDOM, and ABSTRACT—to compare combined performance, task recognition, and task learning.
3 Experimental Setup
The experiments cover diverse classification tasks and three LLM families, using controlled prompts, sampled demonstrations, and varying model sizes and label representations.
- The benchmark contains 16 classification datasets spanning sentiment, toxicity, inference or paraphrase, and topic or stance classification.
- The study evaluates GPT-3, LLaMA, and OPT across models ranging from hundreds of millions to tens of billions of parameters.
- For each test example, demonstrations are sampled from training data, and results average across datasets and three prompt templates.
- The largest OPT-175B model is omitted because of computational constraints.
- Task Setup: Natural prompts resemble prior manual prompts, whereas minimal prompts remove task instructions for ABSTRACT.
- Task Setup: ABSTRACT tests numeric, alphabetic, and symbolic class labels to evaluate mappings represented by abstract characters.
4 Results
Across GPT-3, LLaMA, and OPT, the experiments separate task recognition from task learning by comparing GOLD, RANDOM, and ABSTRACT settings. Task recognition remains broadly available across scales, while task learning improves with model size and demonstrations, especially on simpler tasks.
- Overall trends: GOLD performs best overall, while RANDOM remains largely flat as model size and demonstration count increase.With small models or K = 8, RANDOM is close to GOLD; ABSTRACT increasingly improves with scale and demonstrations.
- Task recognition: Around 10 points for GPT-3 ada and 5 points for OPT-350M separate task recognition from the random baseline with only 8 examples.Even the smallest 350M-parameter models recognize tasks, and this advantage does not drastically scale with model size or example count.
- Task learning: With only 16 examples, OPT-66B and davinci match GOLD performance while using a new label mapping.LLaMA-65B shows the same improvement trend with larger models or more demonstrations, although its ABSTRACT result is less competitive with GOLD.
- Task learning: Numbers and letters consistently outperform symbols across ABSTRACT labels on davinci and OPT-66B.The authors attribute this difference to the greater frequency of letters and numbers in pre-training, making them more natural label spaces.
- Task difficulty: ABSTRACT scales better on simpler sentiment-analysis tasks than on more difficult natural-language-inference tasks.The flatter NLI ABSTRACT curve suggests greater reliance on natural prompts and pre-training priors.
5 Related Work
Prior ICL research explains performance through pre-training concepts, distributional properties, implicit optimization, and mechanistic pattern-recognition circuits. This paper extends comparisons with controlled analyses across model scales, demonstration counts, and public as well as closed-source model families.
- Pre-training accounts: Xie et al. frame ICL as implicit Bayesian inference over pre-trained latent concepts, while Min et al. show that limited information can suffice without true mappings.This motivates separating task recognition from task learning.
- Pre-training accounts: Pre-training distribution properties such as burstiness, label multiplicity, long-tailed classes, and term frequency are linked to ICL behavior.These properties are characteristic of natural language and term frequency positively correlates with model performance.
- Optimization and mechanisms: Other theories describe ICL as implicit gradient descent, with empirical work finding similarities between in-context learning and explicit fine-tuning.Mechanistic studies additionally identify induction heads involved in in-context pattern recognition.
- Positioning: Compared with earlier work, this study examines how GOLD, RANDOM, and ABSTRACT behavior changes with model size and demonstration count.It finds that the GOLD–RANDOM deficit grows with scale and evaluates GPT-3, LLaMA, and OPT, including public models.
6 Conclusion
The paper argues that ICL comprises two distinct phenomena: task recognition and task learning, which emerge under different conditions. It concludes that future ICL studies should distinguish them and report the conditions under which experiments are conducted.
- Conclusion: Task recognition is available even to small models but does not substantially scale with model size or demonstration count.This contrasts with task learning, which requires larger models.
- Conclusion: Task learning emerges in large models and improves with more demonstrations, whereas small models remain unable to learn new mappings even with additional examples.The conclusion treats recognition and learning as distinct abilities rather than one undifferentiated ICL capability.
- Implication: Future ICL research should distinguish task recognition from task learning and clearly state the experimental conditions.The recommendation follows from their different emergence patterns.
Limitations
The study is limited to classification tasks, leaving other NLP task types for future work, and its account of task learning remains mechanistically unresolved.
- The experiments cover classification tasks because they are easier to adapt to the RANDOM and ABSTRACT settings, leaving other NLP tasks for future work.
- The study observes novel mappings to abstract labels but does not determine how models mechanistically learn them.
- Possible mechanisms include implicit gradient descent, mapping demonstrations to pretrained concepts, or an advanced form of task recognition.
A Datasets
The study uses 16 datasets spanning four classification task types and evaluates prompts sampled from training data on development-set examples.
- The benchmark contains 16 datasets across sentiment analysis, toxicity detection, natural language inference/paraphrase detection, and topic/stance classification.
- Prompts are constructed by sampling from each dataset’s training set and evaluated on the development set.
B Prompt Templates
The prompt designs distinguish natural-language task information from abstract-label conditions while keeping responses directly evaluable as single tokens.
- Three natural-language templates are manually designed for each classification task and applied to GOLD and RANDOM label sets.
- ABSTRACT uses task-agnostic minimal templates to avoid exposing task information through prompt wording.
- All prompts require single-token responses, allowing direct evaluation of the models’ last-token predictions without decoding.
C More Results
Across model families, ABSTRACT performance improves with model scale and additional demonstrations, whereas RANDOM changes little; task-type breakdowns show the same overall pattern despite greater noise.
- Small models struggle with ABSTRACT, while the largest models, especially GPT-3 Davinci and OPT-66B, perform it successfully.
- Larger models’ ABSTRACT performance improves further when more demonstrations are provided.
- RANDOM performance changes little as demonstrations increase, whereas ABSTRACT improves drastically, especially for GPT-3 Davinci and OPT-66B.
- Across task-type breakdowns, ABSTRACT scales better with model size and number of examples than RANDOM, although individual results are noisier.
- The results are organized across 16 datasets, model families, demonstration counts, and prompt or label configurations in figures and accuracy tables.