Source-linked AI summary
Are Emergent Abilities of Large Language Models a Mirage?
Rylan Schaeffer, Brando Miranda, Sanmi Koyejo
TL;DR
The paper asks whether sharp, unpredictable LLM emergent abilities reflect fundamental changes in model behavior or researchers’ measurement choices. It presents a mathematical explanation and tests it across language-model benchmarks and vision tasks, finding that metric changes can make the phenomenon disappear or appear.
Problem
The paper examines claims that LLM abilities absent in smaller models appear sharply and unpredictably in larger models, raising questions relevant to AI safety and alignment.
Method
The paper develops a simple mathematical model and tests predictions using InstructGPT/GPT-3, BIG-Bench analyses, and metric changes across vision tasks and architectures.
Results
Emergent abilities appeared with 4/39 metrics, while Multiple Choice Grade and Exact String Match accounted for > 92% of claimed cases.
Takeaways & Limitations
For fixed tasks and model families, researchers can create or ablate apparent emergent abilities by changing evaluation metrics.
Takeaways & Limitations
For model families that are not publicly queryable, the analysis is limited to published results because outputs are unavailable.
Abstract
from arXiv · showhide
Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities intriguing is two-fold: their sharpness, transitioning seemingly instantaneously from not present to present, and their unpredictability, appearing at seemingly unforeseeable model scales. Here, we present an alternative explanation for emergent abilities: that for a particular task and model family, when analyzing fixed model outputs, emergent abilities appear due to the researcher's choice of metric rather than due to fundamental changes in model behavior with scale. Specifically, nonlinear or discontinuous metrics produce apparent emergent abilities, whereas linear or continuous metrics produce smooth, continuous predictable changes in model performance. We present our alternative explanation in a simple mathematical model, then test it in three complementary ways: we (1) make, test and confirm three predictions on the effect of metric choice using the InstructGPT/GPT-3 family on tasks with claimed emergent abilities; (2) make, test and confirm two predictions about metric choices in a meta-analysis of emergent abilities on BIG-Bench; and (3) show to choose metrics to produce never-before-seen seemingly emergent abilities in multiple vision tasks across diverse deep networks. Via all three analyses, we provide evidence that alleged emergent abilities evaporate with different metrics or with better statistics, and may not be a fundamental property of scaling AI models.
1 Introduction
Emergence describes properties that appear as complex systems increase in scale, and LLM research applied this idea to abilities absent in smaller models but present in larger ones.
- Emergence describes new properties that may materialize as system complexity increases.
- Emergent abilities are defined as abilities absent in smaller-scale models but present in large-scale models.
- The concept was first associated with GPT-3 and later discussed as abrupt, task-specific performance changes at scale.
1. Sharpness, transitioning seemingly instantaneously from not present to present
The paper questions whether LLM emergent abilities reflect fundamental model changes, proposing instead that metrics and limited statistical resolution can create their apparent sharpness and unpredictability.
- Emergent abilities are characterized by sharp transitions and unpredictable model scales as performance increases.
- These claims matter because they raise concerns that larger models could acquire dangerous capabilities without warning.
- The paper proposes that nonlinear or discontinuous metrics can create apparent emergence from smoothly changing per-token error rates.
- The authors test this explanation with a mathematical model, InstructGPT/GPT-3 predictions, BIG-Bench meta-analysis, and metric manipulation across vision tasks.
2 Alternative Explanation for Emergent Abilities
The paper models smoothly improving per-token performance and shows how metric choices can transform it into apparent emergence, while alternative metrics reveal predictable scaling.
- The proposed mechanism begins with smoothly, continuously, and predictably declining test loss as model parameter count increases.The illustrative assumption uses power-law cross-entropy scaling, but the authors do not require that specific functional form.
- A metric requiring all L output tokens to be correct makes performance scale nonlinearly with sequence length.For integer addition, exact output matching receives a score of 1 only when every target digit is correct.
- Token Edit Distance reveals smooth, continuous, and predictable improvements where Accuracy produces sharp apparent emergence.
- Brier Score removes apparent emergence produced by discontinuous Multiple Choice Grade.
- The explanation also includes insufficient resolution for smaller models and insufficient sampling of larger models.The stated resolution is set by 1/test dataset size.
3 Analyzing InstructGPT/GPT-3’s Emergent Arithmetic Abilities
The arithmetic experiments test whether apparent emergent abilities depend on metric choice and measurement resolution rather than abrupt changes in model outputs. Across fixed InstructGPT/GPT-3 outputs, linear metrics and higher-resolution accuracy measurements reveal smooth, predictable scaling behavior.
- Increasing target length from 1 to 5 decreases Token Edit Distance performance in an approximately quasilinear, predictable manner.This confirms the first half of the target-length prediction.
- The study tests three predictions using 2-shot multiplication and addition tasks from the InstructGPT/GPT-3 family.The tasks are multiplication between two 2-digit integers and addition between two 4-digit integers.
- Changing from nonlinear Accuracy to linear Token Edit Distance reveals smooth, continuous, predictable improvement with increasing model scale.This holds while keeping the models’ outputs fixed, supporting metric choice as the source of the apparent emergence.
- With Accuracy, apparent emergent abilities occur on both arithmetic tasks when targets have 4 or 5 digits.Accuracy is a nonlinear metric whose effects can make performance appear sharp and unpredictable.
- Increasing test data reveals above-chance Accuracy for all models, with smooth, continuous, predictable improvement and approximately geometric decay as target length increases.The higher-resolution results qualitatively match the mathematical model and confirm the better-statistics prediction.
4 Meta-Analysis of Claimed Emergent Abilities
The BIG-Bench meta-analysis finds that claimed emergent abilities cluster under a small set of nonlinear or discontinuous metrics rather than consistently appearing across task-model family pairs. Changing the metric from discontinuous Multiple Choice Grade to continuous Brier Score removes LaMDA’s apparent emergence.
- Predictions and data: The analysis tests predictions that emergence should concentrate on specific metrics and vanish when those metrics are replaced by linear or continuous alternatives.The study uses published BIG-Bench results and focuses on LaMDA because its outputs are available through BIG-Bench.
- Metric concentration: At most 5 of BIG-Bench’s 39 preferred metrics display possible emergent abilities, and many are nonlinear or discontinuous.Examples include Exact String Match, Multiple Choice Grade, and ROUGE-L-Sum.
- Metric concentration: Hand-annotated task-metric-model family triplets show emergence under only 4/39 metrics, with Multiple Choice Grade and Exact String Match accounting for > 92% of claimed emergent abilities.Multiple Choice Grade is discontinuous, while Exact String Match is nonlinear.
- Metric changes: LaMDA’s emergent abilities under discontinuous Multiple Choice Grade disappear when the same tasks are evaluated with continuous Brier Score.Brier Score is a strictly proper scoring rule for mutually exclusive outcomes.
5 Inducing Emergent Abilities in Networks on Vision Tasks
The paper induces seemingly emergent abilities in vision networks by changing evaluation metrics, despite smooth performance under conventional measures. The demonstrations span autoencoders and autoregressive Transformers across reconstruction and classification tasks.
- Cross-architecture demonstrations: Metric choice induces seemingly emergent abilities across fully connected, convolutional, and self-attentional networks on vision tasks.The authors use vision because abrupt capability transitions had not been observed there to their knowledge.
- Image classification: Autoregressive Transformers classify Omniglot images with increasing accuracy as scale increases, while redefining accuracy as correctly classifying all images creates a seemingly emergent ability.The subset-accuracy measure assigns 1 only when all L images in a sequence are classified correctly.
- Image reconstruction: Shallow nonlinear autoencoders show smoothly decreasing squared reconstruction error as bottleneck units increase, but Reconstructionc produces a sharp, seemingly unpredictable ability for particular thresholds c.The metric counts test data with squared reconstruction error below threshold c.
6 Related Work
Related work includes explanations treating emergent abilities as real changes in scaling laws or as consequences of data assumptions. This paper instead formalizes metric choice as an alternative explanation and reports evidence across benchmarks and vision models.
- Metric explanations: Prior work hypothesized that metrics may partly explain emergence because accuracy can appear sharp while cross entropy does not.This paper turns that discussion into precise predictions and quantitative tests.
- Reported findings: The paper reports that metric choice is likely wholly responsible for observed emergent abilities across benchmark and vision analyses.It also reports graded improvements under widely used metrics and frequent emergence under discontinuous Multiple Choice Grade.
- Alternative accounts: Other work explains emergence through a piece-wise power law or strong data assumptions, whereas this paper argues researcher-induced emergence can occur under a single power law.These accounts differ in whether the underlying emergent change is treated as real or measurement-induced.
7 Discussion
The discussion argues that metric selection can create or remove apparent emergent abilities for fixed tasks and model families, so these abilities may not be fundamental properties of scaling models. It also emphasizes measurement controls, statistical resolution, and public access to models and outputs.
- Interpretation: For a fixed task and model family, choosing different metrics can create or ablate an apparent emergent ability.The authors caution that this does not establish that large language models cannot display genuine emergent abilities.
- Measurement implications: Benchmark construction should treat task and metric as distinct choices and account for how metrics affect per-token error rates.The paper specifically recommends sufficient data when using accuracy to measure performance accurately.
- Statistical controls: Claims about large-model capabilities require proper controls because multiple comparisons across BIG-Bench task-metric-model family triplets can produce emergence by random chance.BIG-Bench includes at least 220 tasks, approximately 40 metrics per task, and approximately 10 model families.
- Reproducibility boundary: Independent investigation is constrained when models and their generated outputs are not publicly available.The paper identifies public access as important for testing explanations of claimed emergent abilities.
A Approximate Behavior of Metrics on Sequential Data
The paper approximates autoregressive model outputs using a fixed per-token error probability, while acknowledging that this simplification is empirically false and analytically limited.
- The analysis asks how different metrics behave when measuring autoregressive model outputs.The authors describe an approximate answer because exact analysis may be difficult or analytically unsolvable.
- The setup uses N test sequences of length L with target tokens and model-predicted tokens.The model is represented through target and predicted token sequences, with a true but unknown per-token error probability ϵ.
- The model assumes a constant per-token error probability, although real token errors do not satisfy this assumption.Modeling dependencies among errors is outside the paper’s scope.
A.1 Per-Token Error Probability is Resolution-Limited
Performance estimates are resolution-limited by the number of observed tokens, and metric choice determines whether errors appear to scale linearly or sharply with sequence length.
- 1/NL is the smallest interval by which the per-token error probability can be resolved with N sequences of length L.The paper illustrates this resolution limit using coin-flip outcomes and introduces resolution-limited notation.
- Token edit distance counts the additions, deletions, and substitutions needed to transform one token sequence into another.It adapts Levenshtein distance from characters to tokens.
- Expected token edit distance scales approximately linearly with the resolution-limited per-token error probability.The rate is slightly higher than linear because additions and deletions add non-negative cost, though their likelihood is not modeled.
- Accuracy falls approximately geometrically with target token length, making this nonlinear metric especially sensitive to limited resolution.Accuracy forces distinctions among quantities that decay rapidly as target length increases.
- ROUGE-L-Sum measures a union longest common subsequence stitched across candidate and reference sequences.The paper presents it as another BIG-Bench metric based on longest common subsequence structure.
B Inducing Emergent Abilities in Networks on Vision Tasks
The authors induce apparently emergent abilities in vision models by changing evaluation metrics, despite the underlying LeNet accuracy improving smoothly with model size.
- LeNet’s ordinary MNIST test accuracy increases smoothly and predictably as the number of model parameters grows.This provides the baseline behavior before redefining the metric.
- ROUGE-L-Sum can sharply fall when per-token error probability increases from 0.05 to 0.1.The simulation illustrates how a sharp metric can transform a modest error change into a large performance change.
- The MNIST example shows that seemingly unpredictable emergent changes can be induced from predictable network behavior through metric definition.The figure contrasts smooth test accuracy with the newly defined subset-accuracy behavior.
- Subset accuracy, scored as 1 only when K of K independent examples are classified correctly, produces an apparent emergent ability as K increases from 1 to 5.Sparse sampling of model sizes makes the induced transition appear especially emergent.