PaperlayerView exact revision on arXiv

Source-linked AI summary

Are Emergent Abilities in Large Language Models just In-Context Learning?

Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, Iryna Gurevych

arXiv:2309.01809v2cs.CL

TL;DR

The paper examines whether claimed emergent abilities remain after separating them from in-context learning and related prompting effects. Across comparisons of model families, scales, and tuning conditions, it concludes that the tested functional abilities are better explained by implicit in-context learning alongside memory and linguistic knowledge, while noting important scope limitations.

  • Problem

    The paper asks whether functional abilities called emergent persist without prompting techniques such as in-context learning and whether instruction-tuned capabilities are better explained by implicit in-context learning.

  • Method

    The study compares GPT, T5, Falcon, and LLaMA models across scales and contrasts instruction-tuned and non-instruction-tuned conditions with and without in-context examples.

  • Results

    The results confirm that previously observed functional linguistic abilities are consequences of in-context learning and that instruction-tuned abilities more likely reflect implicit in-context learning than emergent abilities.

  • Takeaways & Limitations

    Instruction following does not imply reasoning or latent dangerous abilities, so LLM capabilities should not be overestimated.

  • Takeaways & Limitations

    The study cannot exactly match parameter counts across architectures and assumes no unreported data leakage beyond official publications.

Abstract

from arXiv · show

Large language models, comprising billions of parameters and pre-trained on extensive web-scale corpora, have been claimed to acquire certain capabilities without having been specifically trained on them. These capabilities, referred to as "emergent abilities," have been a driving force in discussions regarding the potentials and risks of language models. A key challenge in evaluating emergent abilities is that they are confounded by model competencies that arise through alternative prompting techniques, including in-context learning, which is the ability of models to complete a task based on a few examples. We present a novel theory that explains emergent abilities, taking into account their potential confounding factors, and rigorously substantiate this theory through over 1000 experiments. Our findings suggest that purported emergent abilities are not truly emergent, but result from a combination of in-context learning, model memory, and linguistic knowledge. Our work is a foundational step in explaining language model performance, providing a template for their efficient use and clarifying the paradox of their ability to excel in some instances while faltering in others. Thus, we demonstrate that their capabilities should not be overestimated.

1 Introduction, Motivation and Context

The paper distinguishes inherent linguistic abilities from prompting techniques such as in-context learning and instruction-tuning. It argues that purported emergent functional abilities may instead reflect prompting, memory, and formal linguistic knowledge, with implications for capability and safety assessments.

  • LLMs are reported to show abilities absent from smaller models, motivating claims about emergent reasoning and other functional linguistic capabilities.
  • The paper narrows its safety conclusion: evidence from current LLM abilities does not support existential-threat concerns, while misuse risks and future systems remain outside that conclusion.
  • In-context learning supplies examples in the prompt so models infer how to perform a task, while instruction-tuning trains models on prompts paired with desired outputs.
  • Because prompting techniques can appear at similar scales yet remain partly predictable in smaller models, their effects must be separated from genuinely emergent abilities.
  • The study asks which abilities remain emergent without in-context learning and whether instruction-tuned models are better explained by implicit in-context learning.
  • The authors claim no emergent functional linguistic abilities without in-context learning and attribute instruction-tuned capabilities to formal skills, information recall, and in-context learning.

2 Experimental Setup

The experiments compare multiple model families and scales across selected tasks in zero-shot and few-shot settings, using several metrics and checks designed to reduce evaluation and prompting biases. The setup separates in-context learning from zero-shot performance while testing whether the evaluation framework reproduces prior findings.

  • The study evaluates 20 models across 22 tasks in two settings, using four metrics and additional bias tests including manual analysis.
  • Four model families—GPT, T5, Falcon, and LLaMA—are sampled at multiple parameter scales to assess abilities in relation to model size.
  • The task set includes 14 of 17 previously emergent BIG-bench tasks, plus randomly selected baseline tasks; memorisability is manually assessed from 50 examples per task.
  • Few-shot evaluation uses five in-context examples, whereas zero-shot evaluation omits them and therefore does not explicitly use in-context learning.
  • Evaluation combines exact match accuracy, BERTScore accuracy, and string edit distance to test whether conclusions depend on output format or discrete metrics.
  • Prompt refinements, shortened outputs, and manual checks are used to reduce disadvantages and confounds for non-instruction-tuned models.

3 Emergence in GPT in the Absence of In-Context Learning

In zero-shot, non-instruction-tuned GPT models show little evidence of emergent functional abilities once in-context learning is excluded. Only two tasks meet the emergence criteria, and both involve formal knowledge or recall rather than reasoning; the broader pattern generalizes across model families.

  • Zero-shot prompting excludes in-context examples, enabling assessment of emergent abilities in non-instruction-tuned GPT models.
  • Only two tasks meet the paper’s emergence criteria when in-context learning is controlled, while Misconceptions and Strategy QA show only marginal unpredictable improvements.
  • The two identified tasks are Nonsense words grammar and Hindu knowledge; neither requires reasoning, involving formal linguistic ability or information recall instead.
  • No functional linguistic abilities emerge in the non-instruction-tuned 175B GPT model davinci without in-context learning.
  • Across additional model families, performance is either predictable from smaller models or below baseline, supporting generalisability of the findings.

4 Instruction-Tuning as Implicit In-Context Learning

The experiments compare instruction-tuned and explicitly in-context learning models to test whether instruction-tuning provides distinct functional abilities or better access to in-context capabilities. Their substantial task overlap, including 16 of 22 tasks in a larger-scale comparison, supports implicit in-context learning as the proposed explanation.

  • 4.1 Comparative Analysis of Initial Tasks: The comparison tests whether GPT-J’s few-shot-solvable tasks overlap with Flan-T5-large’s zero-shot-solvable tasks.Flan-T5-large shows no zero-shot-to-few-shot performance change, whereas GPT-J improves with few-shot prompting.
  • 4.1 Comparative Analysis of Initial Tasks: Despite major architectural, training-data, tuning, and scale differences, the models substantially overlap in above-baseline tasks and performance scores.The comparison excludes Modified arithmetic because that task requires in-context demonstrations.
  • 4.1 Comparative Analysis of Initial Tasks: Five of 21 tested tasks show a marked above-baseline advantage for one model without the other.Hindu knowledge favors the larger GPT-J, while Codenames is especially challenging for the non-instruction-tuned model.
  • 4.1 Comparative Analysis of Initial Tasks: Figure 2 shows that the two models perform above the random baseline on substantially overlapping task sets, favoring effective access to in-context capabilities.The figure interprets this overlap as evidence against emergent functional linguistic abilities.
  • 4.2 Generalisability: The larger-scale zero-shot comparison asks whether scale or program-code training changes which tasks exceed baseline.Restricting evaluation to zero-shot settings avoids explicit in-context capabilities that increase with scale.
  • 4.2 Generalisability: Sixteen of 22 tasks show congruent above-baseline performance between Flan-T5-large and text-davinci models, supporting consistency across scale and tuning-data differences.Some non-overlapping recall tasks favor the larger GPT models.
  • 4.3 A Novel Theoretical Foundation: Instruction-tuning may enable models to map instructions into a form suitable for implicit in-context learning.The authors leave the exact mechanism of this mapping for future work.

5 Related Work

Related work defines emergent abilities through above-random performance that appears only in larger models, while subsequent research questions whether such abilities truly emerge. In-context learning is presented as prompting models with demonstrations, with theoretical work linking it to implicit learning mechanisms and pretraining-data distributions.

  • Emergent Abilities: Emergent abilities were initially defined as abilities absent in smaller models but present in larger models.Prior reviews identified such abilities using above-random performance on benchmark tasks.
  • Emergent Abilities: Prior studies reported additional candidate emergent abilities, including Theory of Mind and cognitive biases, while other work questioned their existence.The cited literature spans GPT-3, PaLM, Chinchilla, Gopher, LaMDA, BIG-bench, and MMLU.
  • In-Context Learning: In-context learning prompts an LLM with demonstrations so it can infer how to perform a task and answer accordingly.The literature describes ICL as an increasingly prominent paradigm for large language models.
  • In-Context Learning: Theoretical studies relate ICL to fine-tuning, implicit gradient descent, inference-time function construction, gradient-based meta-learning, and pretraining-data distributions.These accounts provide several proposed mechanisms for how demonstrations influence model behavior.

6 Conclusions and Implications

The paper reports that its two hypotheses were confirmed: previously observed functional linguistic abilities are consequences of ICL, and instruction-tuned capabilities more likely reflect implicit ICL. It distinguishes instruction following from inherent problem-solving ability, with implications for interpreting hallucinations and using LLMs more efficiently.

  • 6 Conclusions and Implications: The authors hypothesized that previously observed functional linguistic abilities result from ICL and that instruction-tuned abilities more likely reflect implicit ICL.They report that both hypotheses were confirmed.
  • 6 Conclusions and Implications: Following instructions does not imply reasoning ability or latent dangerous capabilities.The paper connects this distinction to fluent but factually incorrect hallucinated output.
  • 6 Conclusions and Implications: Understanding the distinction between instruction following and inherent problem solving informs how LLMs are used and what problems they are assigned.The authors present this distinction as significant for methods of utilizing LLMs.
  • 6 Conclusions and Implications: The paper aims to demystify LLM abilities and limitations, alleviate related safety concerns, and provide a framework for more efficient use.These implications are stated as consequences of developing a deeper understanding of model behavior.

Limitations

The evaluation spans many model sizes and architectures but cannot exactly match parameter counts across architectures. It also assumes no unreported data leakage and is limited to English tasks.

  • Parameter counts were not exactly matched across T5, GPT, Falcon, and LLaMA because publicly available releases vary.The study used the parameter counts available for each model rather than conducting new pre-training for matched sizes.
  • The analysis assumes that data leakage did not occur beyond what official model publications reported.Unreported leakage could bias performance and the classification of tasks as memory-based.
  • The experiments are limited to English tasks, primarily because prior work and computational budgets constrained evaluation.

Ethical Considerations

The paper distinguishes its conclusions about emergent functional abilities from broader risks of LLM misuse and AI safety. Its experiments use carefully designed prompts and validation tasks to assess whether apparent performance reflects genuine emergence.

  • Ethical Considerations: The authors do not claim that LLMs have no potential for harm, citing convincing fake news and phishing as continuing risks.They call for detection mechanisms, ethical guidelines, and continued AI safety research.
  • Ethical Considerations: The study frames its findings as an opportunity to prioritize pressing safety concerns while exploring research beyond scaling alone.
  • Prompting and Evaluation: The experiments attempt to mitigate evaluation biases and, when mitigation is impossible, maximize the chance of detecting genuine emergent abilities.
  • Prompting and Evaluation: Completion-style prompts are refined so non-instruction-tuned models can solve tasks without relying on explicit instruction comprehension.
  • Prompting and Evaluation: The study compares closed prompts with answer choices, open prompts without choices, and adversarial prompts requiring alternate output choices.Open prompts often produced outputs unrelated to the answer choices, motivating the use of more controlled formats.
  • Results: Among seven above-random tasks for non-instruction-tuned GPT-3, three were predictable from smaller models and therefore not considered emergent.
  • Results: Physical intuition was the only task with a notable, unpredictable improvement over baseline in that analysis.
  • Results: Common morpheme showed no evidence of emergent functional linguistic ability because its 50-example test set made the accuracy improvement a small fraction of the total.

B.3 Manual Evaluation of Responses

Manual evaluation of 50 examples per task tested whether automatic scoring distorted conclusions about emergent abilities. BERTScore generally overestimated performance, while lenient manual scoring did not change the conclusion that notable gains were not emergent.

  • 50 examples from each task were manually evaluated against BERTScore accuracy to check whether automatic scoring affected conclusions.BERTScore accuracy and manual evaluation accuracy were compared on the same examples.
  • In exactly three cases, manual evaluation increased performance, but two increases were only marginally above baseline.Logical deduction involved copied answers that were technically correct, while Causal judgement rose only slightly above its 50% baseline.
  • BERTScore generally overestimated model performance in cases with notable disagreement from manual evaluation.The manual analysis found automatic scoring produced false positives in the majority of such cases.
  • Commonsense QA showed the only marked increase over baseline, but its performance was predictable from smaller models and therefore not emergent.
  • Despite imprecision from evaluating only 50 examples per task, the manual analysis did not fundamentally alter the study’s conclusions.
  • Non-instruction-tuned models qualitatively interpreted task requirements, including producing the requested yes-or-no answers for Causal judgement.

C Experimental Setup

The experimental setup evaluates model families across selected task sets using multiple prompting strategies and repeated trials. Scoring uses BERTScore for semantic similarity and task-specific random baselines for comparison.

  • GPT and T5 models were evaluated on all 22 tasks, while Falcon and LLaMA models were evaluated on four representative tasks.The representative tasks included Logical Deductions, Social IQA, GSM8K, and Tracking Shuffled Objects.
  • Each evaluation used closed and adversarial prompting strategies, with Falcon and LLaMA experiments repeated three times to account for response variance.
  • Experiments were generally repeated three times and averaged, while GPT-3 175B API evaluations were run once with temperature 0.Local experiments used temperature 0.01 and batch size 16; API evaluation aimed for deterministic outputs.
  • BERTScore accuracy measures semantic similarity between model outputs and provided answer choices.
  • Random baselines were constructed separately for each task by repeatedly selecting options and averaging the resulting scores.

D Additional Results: Implicit In-Context Learning

Additional results compare several models under completion prompting while distinguishing previously identified emergent tasks from non-emergent ones. Modified arithmetic is excluded because its construction requires in-context demonstrations.

  • Figure 5 compares Flan-T5-large, GPT-J, text-davinci-001, and text-davinci-003 using completion prompting.The comparison includes zero-shot and few-shot settings across the listed models.
  • Modified arithmetic is excluded because the task is constructed to require in-context demonstrations.

E Detailed Task Information

The paper evaluates a broad set of tasks, including benchmarks previously identified as emergent and tasks covering reasoning, social understanding, emotional intelligence, and word association. Task descriptions are paired with representative examples.

  • The task collection includes tasks previously identified as emergent in GPT-3, primarily drawn from BIG-bench.
  • The task listings provide brief descriptions and sample inputs, including Figure of speech detection, Tracking shuffled objects, and example question-answer pairs.
  • Codenames asks models to identify words associated with a given word.
  • Social IQA measures reasoning about the commonsense implications of social situations.
  • Strange stories measures emotional intelligence through a psychology test using naturalistic short stories.
  • Strategy QA tests open-domain questions whose implicit reasoning steps must be inferred from the question.

F Task Memorisability

The section classifies tasks by whether models could achieve above-random performance through memory recall rather than the intended reasoning process. It uses representative examples to distinguish memorisable from non-memorisable tasks and notes a limitation of this classification.

  • Task classification: A task is memorisable when a language model could achieve above-random performance by repeating factual information from memory.This shortcut bypasses the reasoning path intended by the task, so scale-related gains are less likely to indicate emergence.
  • Task classification: The analysis categorises selected tasks by cognitive skill and degree of memorisability.Examples are used to justify the memorisable versus non-memorisable classification.
  • Classification caveat: A highly memorisable task could still involve the intended reasoning process, so performance gains alone cannot establish emergence.The authors therefore treat gains on highly memorisable tasks as less indicative of emergence than gains on non-memorisable tasks.
  • Memorisable examples: Recall-based examples include factual questions such as whether Pollock was trained by Leonardo da Vinci and which Hindu deities belong outside the Trimurti.These examples are presented as solvable by recalling previously encountered information.
  • Non-memorisable examples: Non-memorisable examples require combining known concepts in novel scenarios or tracking relationships among objects.The escapee scenario requires combining concepts, while the book-ordering task requires spatial tracking in a novel setting.
Loading 2309.01809v2…