PaperlayerView exact revision on arXiv

Source-linked AI summary

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers

Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, Furu Wei

arXiv:2212.10559v3cs.CL

TL;DR

The paper asks how GPT-based in-context learning works despite requiring no parameter updates. It models ICL as implicit finetuning through meta-gradients and tests this account against explicit finetuning, finding similar behavior across multiple perspectives; momentum-based attention further supports the view and improves over vanilla attention. The analysis is limited to Transformer-based ICL, models up to 2.7B parameters, and classification tasks.

  • Problem

    Although ICL achieves strong performance from demonstration examples without parameter updates, its working mechanism remains an open question.

  • Method

    The paper treats GPT as a meta-optimizer, deriving meta-gradients from demonstrations through forward computation and applying them through attention, based on a duality with gradient descent.

  • Results

    ICL behaves similarly to explicit finetuning across multiple perspectives, while momentum-based attention consistently outperforms vanilla attention on language modeling and ICL.

  • Takeaways & Limitations

    The results support understanding ICL as implicit finetuning and indicate that this perspective can inform future model design.

  • Takeaways & Limitations

    The empirical analysis covers GPT models up to 2.7B parameters and classification tasks, excluding larger models and tasks such as multiple choice and open-ended generation.

Abstract

from arXiv · show

Large pretrained language models have shown surprising in-context learning (ICL) ability. With a few demonstration input-label pairs, they can predict the label for an unseen input without parameter updates. Despite the great success in performance, its working mechanism still remains an open question. In this paper, we explain language models as meta-optimizers and understand in-context learning as implicit finetuning. Theoretically, we figure out that Transformer attention has a dual form of gradient descent. On top of it, we understand ICL as follows: GPT first produces meta-gradients according to the demonstration examples, and then these meta-gradients are applied to the original GPT to build an ICL model. We comprehensively compare the behaviors of in-context learning and explicit finetuning on real tasks to provide empirical evidence that supports our understanding. Experimental results show that in-context learning behaves similarly to explicit finetuning from multiple perspectives. Inspired by the dual form between Transformer attention and gradient descent, we design a momentum-based attention by analogy with gradient descent with momentum. The improved performance over vanilla attention further supports our understanding from another perspective, and more importantly, shows the potential to utilize our understanding for future model design. The code is available at \url{https://aka.ms/icl}.

1 Introduction

The paper frames in-context learning as implicit finetuning: GPT derives meta-gradients from demonstrations through forward computation and applies them through attention. Experiments compare ICL with explicit finetuning, while momentum-based attention tests whether this view can guide model design.

  • 1 Introduction: ICL is proposed as implicit finetuning, with GPT acting as a meta-optimizer over demonstration examples.The model produces meta-gradients during forward computation and applies them through attention.
  • 1 Introduction: On six classification tasks, ICL and explicit finetuning behave similarly across predictions, attention outputs, and attention weights.The comparisons include weights assigned to query tokens and training tokens.
  • 1 Introduction: Momentum-based attention treats attention values as meta-gradients and applies a momentum mechanism by analogy with gradient descent with momentum.Experiments evaluate the design on language modeling and in-context learning.
  • 1 Introduction: Momentum-based attention consistently outperforms vanilla attention on language modeling and in-context learning.The improvement provides additional support for viewing Transformer attention as meta-optimization.
  • 1 Introduction: Transformer attention and gradient descent have a dual form, linking ICL's forward computation with finetuning's back-propagated updates.This dual view motivates interpreting ICL as a form of meta-optimization.

2 Background

The paper defines GPT-based classification ICL using demonstration examples prepended to a query and selects the candidate label with the highest probability. It then motivates a duality between gradient-optimized linear layers and linear attention.

  • 2 Background: GPT-based classification ICL conditions label prediction on a query and n demonstration input-label pairs.The demonstrations are formatted with a predefined template and prepended before the query input.
  • 2 Background: The model computes each candidate answer's probability conditioned on the contextual demonstrations and query.The output probability is based on the hidden state at the last token position and the candidate's output embedding.
  • 2 Background: The predicted classification label is the candidate answer with the highest probability in the restricted label space.The candidate set is denoted Y.
  • 2 Background: A linear layer optimized by gradient descent can be represented through an initialized matrix and an update matrix.The update matrix accumulates outer products of historic input representations and their error signals.
  • 2 Background: Linear attention has a dual representation in which historic output error signals act as values, historic inputs as keys, and the current input as the query.This correspondence motivates the paper's meta-optimization interpretation of attention.

3 Understanding In-Context Learning (ICL) as Implicit Finetuning

The paper interprets ICL as implicit finetuning: attention over demonstrations creates updates analogous to gradient descent, while explicit finetuning applies back-propagated updates. It compares both procedures under matched conditions and identifies several shared properties.

  • 3.1 Understanding Transformer Attention as Meta-Optimization: Transformer attention has a dual form of gradient descent that motivates viewing ICL as meta-optimization.The analysis first considers attention under a relaxed linear form before deriving the dual relationship.
  • 3.1 Understanding Transformer Attention as Meta-Optimization: ICL treats demonstration-dependent attention values as meta-gradients that produce updates to zero-shot parameters.The attention to demonstration tokens is equivalent to parameter updates ΔW_ICL applied to W_ZSL.
  • 3.1 Understanding Transformer Attention as Meta-Optimization: The proposed interpretation has three stages: GPT acts as a meta-optimizer, generates meta-gradients through forward computation, and applies them through attention.These stages build an ICL model without changing the pretrained model parameters.
  • 3.2 Comparing ICL with Finetuning: The comparison restricts explicit finetuning to key and value projection parameters and matches ICL demonstrations, order, formatting, and one-step training.The finetuning updates are obtained by back-propagation from task-specific training objectives.
  • 3.2 Comparing ICL with Finetuning: ICL and finetuning share attention-limited effects, gradient-descent updates, training examples, and causal ordering.ICL produces meta-gradients by forward computation, whereas finetuning obtains gradients through back-propagation.
  • 3.2 Comparing ICL with Finetuning: These common properties support understanding ICL as implicit finetuning and motivate quantitative comparisons between the two procedures.The paper presents this comparison as empirical support for the proposed interpretation.

4 Experiments

Experiments compare ICL with explicit finetuning across six classification datasets and multiple behavioral measurements. Across prediction, representation, and attention patterns, ICL consistently resembles finetuning.

  • 4.2 ICL and Finetuning Improve Performance: Compared with zero-shot learning, both ICL and finetuning achieve considerable validation-accuracy improvements across the six datasets.Table 1 reports validation accuracy for ZSL, finetuning, and ICL settings.
  • 4.3 ICL Covers Most of Correct Predictions of Finetuning: ICL correctly predicts more than 85% of examples that finetuning corrects over zero-shot learning, on average across two GPT models and six datasets.Rec2FTP measures how much finetuning-correct behavior ICL covers from the model-prediction perspective.
  • 4.4 ICL Tends to Change Attention Outputs in the Same Direction as Finetuning: ICL updates are much more similar to finetuning updates than to random updates, indicating aligned attention-output representation changes.SimAOU compares cosine similarity between ICL and finetuning updates against similarity between ICL and randomly generated updates.
  • 4.5 ICL Behaves Similarly to Finetuning in Attention: ICL generates attention weights more similar to post-finetuning weights than to pre-finetuning weights across the evaluated models and datasets.SimAM measures similarity of attention maps to query tokens before and after finetuning.
  • 4.6 ICL and Finetuning Tend to Pay Similar Attention to Training Tokens: Kendall (ICL, FT) remains distinctly positive while Kendall (ICL, Random) stays near zero, showing similar attention rankings over training tokens.The comparison is averaged across examples and layers for two GPT models on six datasets.

5 Momentum-Based Attention Inspired by Dual Form of Transformer Attention

The paper introduces momentum into Transformer attention by analogy with momentum-based gradient descent and evaluates the resulting mechanism. Momentum-based attention consistently improves language-modeling perplexity and in-context-learning performance over vanilla attention.

  • 5 Momentum-Based Attention Inspired by Dual Form of Transformer Attention: The dual form between Transformer attention and gradient descent motivates introducing momentum into attention as an optimization-inspired design.The paper treats attention values as meta-gradients and applies an optimization analogy to Transformer attention.
  • 5 Momentum-Based Attention Inspired by Dual Form of Transformer Attention: Exponential Moving Average averages attention values to construct momentum-based attention, explicitly strengthening attention’s recency bias.The method is motivated by the role of attention values as meta-gradients.
  • Experiments on In-Context Learning: Introducing momentum improves vanilla Transformer accuracy by 2.8 on average across six in-context-learning datasets.The evaluated datasets cover sentiment analysis, natural language inference, and multi-choice selection.
  • Experiments on Language Modeling: Momentum-based attention consistently improves perplexity over the vanilla Transformer on all three validation input lengths.The language-modeling comparison uses two 350M-parameter GPT models evaluated at input lengths 256, 512, and 1024.
  • Experiments on In-Context Learning: Performance improvements in language modeling and in-context learning further support interpreting Transformer attention as meta-optimization.The authors present these results as evidence for the momentum deduction and their broader understanding of attention.

6 Related Work

Related work explains ICL through latent-concept inference, induction heads, or learned algorithms. This paper instead focuses on ICL as a meta-optimization process, with empirical and theoretical links to explicit learning algorithms.

  • Related Work: Implicit Bayesian inference explains ICL as recovering a shared latent concept among demonstrations learned during pretraining.This account emphasizes the latent concept represented by the demonstrations.
  • Related Work: Induction-head analyses attribute ICL to Transformer modules that refer to abstract patterns in previous sequences to predict the next token.These studies focus on specific Transformer modules rather than the learning algorithm of ICL.
  • Related Work: The paper differs by concentrating on ICL’s learning algorithm and explaining it as meta-optimization.This positions the paper’s perspective relative to latent-concept, induction-head, and regression-based accounts.
  • Related Work: Other studies construct or train Transformers to learn linear functions in context, match least-squares estimation, or implement gradient-descent-based learning algorithms.These results concern regression tasks and explicit learning-algorithm behavior.

7 Conclusion

The paper explains GPT-based in-context learning as meta-optimization, theoretically relating Transformer attention to gradient descent and empirically comparing ICL with finetuning. It also reports consistent gains from momentum-based attention over vanilla attention.

  • Transformer attention has a dual form of gradient descent, motivating an interpretation of ICL as meta-optimization.
  • GPT-based ICL is interpreted as implicit finetuning, with demonstration examples producing meta-gradients through forward computation.
  • Across six real NLP classification tasks, ICL behaves similarly to explicit finetuning from multiple perspectives.
  • Momentum-based attention consistently improves performance over vanilla attention in language modeling and ICL experiments.

Limitations

The analysis is scoped to Transformer-based ICL and uses a relaxed linear-attention form for qualitative analysis. Experiments are further limited by computational costs to models up to 2.7B parameters and classification tasks.

  • The paper studies Transformer-based ICL, leaving how ICL works in other architectures such as LSTM for future work.
  • The theoretical analysis uses a relaxed form of linear attention, while standard Transformer attention may have a more complex mechanism.
  • Intermediate-result storage and computational costs limit empirical analysis to GPT models with up to 2.7B parameters.
  • The empirical analysis considers classification tasks but not multiple-choice or open-ended generation.

Appendix

The appendix documents the templates used to format examples and the candidate answer sets for the six classification datasets.

  • Table 8 presents the templates for formatting examples and the candidate answer sets used across six classification datasets.

B Hyper-Parameters for In-Context Learning and Finetuning

The experiments select ICL random seeds and finetuning learning rates through grid search, with details reported for two GPT models across six classification datasets.

  • Grid search selects the best random seed for ICL and the best learning rate for finetuning.
  • The ICL random-seed search uses values 1 through 7, while finetuning searches 36 learning-rate values formed from nine bases and four scales.
  • GPT 1.3B finetuned on SST5 uses learning rate 0.00016 after the standard search failed to outperform zero-shot learning.
  • Table 9 reports the selected random seeds and learning rates for two GPT models on six classification datasets.

C Hyper-Parameters for Training Language Models from Scratch

This appendix section organizes training and evaluation configuration details across several tables. It covers dataset formatting and candidate answers, dataset-specific seeds and learning rates, and scratch-training hyperparameters for two language models.

  • Table 10 summarizes the hyper-parameters used to train two language models from scratch.
  • Table 8 lists formatting templates and candidate answer sets for six classification datasets.
  • Table 9 records selected random seeds and learning rates for two GPT models across six classification datasets.
Loading 2212.10559v3…