PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Language Models

Michael R. Douglas

arXiv:2307.05782v2cs.CLhep-thmath.HOphysics.comp-ph

TL;DR

The lectures address how LLMs can perform intelligent tasks despite being trained for next-word prediction, and survey their history, transformer architecture, capabilities, and interpretability research. They synthesize scaling results, probing methods, theoretical perspectives, and open questions while emphasizing that understanding LLMs remains incomplete.

  • Problem

    LLMs perform diverse intelligent tasks, but how next-word prediction produces these capabilities and how their internal workings can be understood remain open questions.

  • Method

    The lectures survey LLM history and scaling, explain transformer architecture, and review probing, reverse engineering, and theoretical approaches to their internal computations.

  • Results

    Power-law scaling links language-model performance to model size and compute across many decades, supporting better performance from larger models.

  • Takeaways & Limitations

    Understanding LLMs requires explaining how one system learns from diverse training data to perform a wide range of tasks.

  • Takeaways & Limitations

    There is no guarantee that current interpretability procedures will yield an understanding of what LLMs have learned.

Abstract

from arXiv · show

Artificial intelligence is making spectacular progress, and one of the best examples is the development of large language models (LLMs) such as OpenAI's GPT series. In these lectures, written for readers with a background in mathematics or physics, we give a brief history and survey of the state of the art, and describe the underlying transformer architecture in detail. We then explore some current ideas on how LLMs work and how models trained to predict the next word in a text are able to perform other tasks displaying intelligence.

1 Introduction

These notes introduce LLMs through ChatGPT’s rapid public impact and survey their architecture, capabilities, history, and unresolved questions about how next-word prediction produces intelligent behavior.

  • ChatGPT’s natural-language fluency in answering questions, translating, dialoguing, and writing code marked a major advance despite remaining below human abilities.
  • The rapid public impact of ChatGPT makes understanding, forecasting, and governing AI capabilities and limitations urgent.
  • The lectures target mathematically knowledgeable readers who are not necessarily experts in machine learning or artificial intelligence.
  • The central scientific puzzle is how models trained to predict each next word can perform tasks such as solving quantitative word problems.
  • Current explanations range from rearranging training sentences to learning sophisticated internal structures, but the mechanism remains poorly understood.

2 Symbolic and connectionist AI

This section contrasts symbolic and connectionist AI, then explains how data-driven neural methods became dominant as computational power and datasets expanded.

  • AI pursues computational systems capable of tasks requiring language, reasoning, and planning, including theorem proving and question answering.
  • Symbolic AI: Symbolic AI encodes detailed task understanding through explicit rules, formal logic, language theory, search, and heuristics.
  • Connectionist AI: Connectionist AI instead draws on brain hypotheses, information theory, statistics, physics, mathematics, and optimization.
  • Connectionist AI: Deep learning uses large-parameter neural networks that can generalize despite overfitting concerns, while their interpretability remains a key issue.
  • Recent development: Transformer language models trained to predict the next word have revolutionized natural-language processing and achieved remarkable performance across tasks.
  • Recent development: The section attributes this progress partly to exponential growth in computational power and training datasets, which reduced the advantage of hand-coded human understanding.

3 Language models

Language models represent text statistically, generating sequences through conditional word probabilities and training by minimizing cross entropy. The section traces their development from n-gram and recurrent models to transformers, scaling laws, emergent capabilities, and persistent reliability problems.

  • Statistical language models encode a probability distribution over word strings that approximates the distribution in a large text corpus.
  • Generative language models sample missing or subsequent words from conditional distributions, extending text one word at a time.Autoregressive factorization expresses a sequence probability as successive conditional probabilities.
  • Cross entropy sums per-word prediction losses and can be minimized as a network objective; exp −L is also called perplexity.Lower L indicates better models, and standard machine-learning optimization tools train the network.
  • Transformer models introduced attention and positional encoding, while GPT’s next-word prediction approach eventually prevailed over masked-word prediction for generative use.Earlier systems commonly pretrained on word prediction and then fine-tuned on smaller labeled task datasets.
  • Power-law scaling links language-model performance to model size, dataset size, and compute, supporting larger models and informing dataset and hyperparameter choices.The reported exponents are small, approximately −0.076 to −0.095, but the scaling holds across many model-size decades.
  • Improved word-prediction performance coincided with capabilities such as code generation and in-context learning, but LLMs still invent facts and struggle with symbolic reasoning.In-context learning uses examples in the prompt without changing model parameters or performing additional task training.

4 Phenomenology of language models

This section surveys how LLM capabilities are evaluated and examines scaling laws, while emphasizing that links between language-model loss and intelligent-task performance remain unsettled.

  • Evaluation: Benchmarking evaluates LLM accuracy reproducibly across standardized tasks, including translation, question answering, puzzles, classification, summarization, and common-sense reasoning.BIG-bench combines 204 language tasks, while HELM also measures repetition of copyrighted material, bias, and toxicity.
  • Evaluation: Mathematical and scientific reasoning remains incomplete: as of mid-2023, systems find 20–80% of theorem-proving proofs but miss some seemingly easy cases.Related benchmarks test negation, consistency, and compositionality.
  • Evaluation: Synthetic toy worlds are used for theoretical analysis because real-world natural-language benchmarks are too complex for direct theoretical treatment.Examples include arithmetic, games, systems of equations, formal-language parsing, and linear regression.
  • Scaling laws: With model size P fixed and dataset size D increasing, test loss scales as L ∼ 1/D; varying one large parameter can instead produce nontrivial power laws.The proposed scaling form fits Figure 2, but the significance of differing exponents is not convincing.
  • Scaling laws: Perplexity’s relation to performance on intelligent tasks was not predictable in advance, and clear post hoc emergence thresholds have not been identified.The text suggests controlling corpus quality, task details, and related variables, possibly in simpler synthetic settings.
  • Training dynamics: Training dynamics often show long plateaus interrupted by steep drops, with interpretations ranging from learning phenomena to Hessian-eigenvalue randomness.The section presents these observations as part of a broader phenomenological study of model capabilities and scaling.

5 Simpler language models

This section builds language models from word statistics and embeddings, then extends finite-context predictors with neural networks and recurrent state to address longer-range dependencies.

  • N-gram models: An N-gram model estimates the next-word conditional probability from counts of preceding and completed word sequences.The simplest models with N = 3 or 4 work better than expected and can be improved by smoothing.
  • N-gram models: N-gram models cannot usefully take large N because the number of possible strings grows exponentially, leaving most longer strings unseen.Even an internet-scale corpus contains only a vanishingly small fraction of likely twenty-word strings.
  • Embeddings: The distributional hypothesis represents word properties and meanings through statistics of neighboring words.This motivates co-occurrence matrices and word embeddings.
  • Embeddings: Word embeddings map words to vectors derived from co-occurrence statistics, optionally compressed using principal component analysis.A corpus defines an N-gram co-occurrence matrix whose columns provide the initial word-vector map.
  • Embeddings: Embedding arithmetic can capture semantic relations: an inner-product prediction for king − man + woman selects queen when p ≳ 100.The text relates this behavior to ratios of co-occurrence statistics.
  • Neural language models: A general autoregressive model maps input-word embeddings through F, produces a prediction vector, and converts it into a word distribution.The distribution is obtained with the inverse map described by the softmax/Boltzmann rule.
  • Neural language models: Replacing F with a feed-forward neural network yields a deep-learning language model whose large-parameter limit can approximate arbitrary functions.The network alternates linear maps and componentwise nonlinearities such as ReLU.
  • Recurrent models: Recurrent neural networks add a state vector updated with each word, providing memory beyond the fixed context window.This addresses the limitation that next-word prediction may require remembering words farther back than L ≈ 100 positions.

6 Recipe for an LLM

This section presents transformers as sequence-to-sequence maps built from attention and feed-forward layers, with positional embeddings restoring word order and parallel attention enabling efficient processing.

  • Architecture: A transformer alternates attention and independently applied feed-forward layers, mapping a list of L input vectors to a list of L output vectors.Residual connections may combine these layers with the identity function.
  • Attention: Attention forms each output from a learned weighted sum of current and previous inputs, allowing every output to select useful earlier representations.Softmax weights determine the contribution of each input, while a learnable bilinear form controls comparisons.
  • Attention: Multiple attention heads run in parallel with different learned transformations, with their outputs concatenated; typically p = Hq.Each head uses a lower-dimensional output space q than the embedding dimension p.
  • Position: Because attention is permutation-invariant over input vectors, positional embeddings are concatenated with word embeddings to encode word order.The positional vectors may use sinusoidal functions or learned parameters.
  • Computation: Transformer attention can be computed in parallel, so with sufficiently many processors its processing time does not grow with window length L.Its total computation still scales as L^2 because each unit can attend to every previous unit.
  • Scale: The largest GPT-3 described has embedding dimension p = 12288, window length L = 4096 or 8192, depth D = 96, and H = 96 heads.Its total parameter count is roughly 12Dp^2.
  • Training: Training starts from random parameters and optimizes the language-model objective over natural-language data, producing an LLM with the described capabilities.The learning rate η is reported as around 10^-4.

7 Studying the internal workings

The notes survey methods for understanding how LLMs represent information, compute functions, and learn those implementations from data. They emphasize both probing internal representations and reverse-engineering circuits, while noting that the connection between trained models and interpretable mechanisms remains difficult.

  • Representations: Embedding maps and contextualized embeddings depend on model details, training data, initialization, data order, and batching, with nonlinear loss landscapes amplifying small differences.
  • Computations: Othello-GPT predicts legal moves while exposing whether its internal representations reflect the board state, despite being trained only on Othello games.The board-state mapping is nonlocal and nonlinear, making it a useful test of internal world-model representations.
  • Representations: Researchers probe intermediate activations to test whether LLMs encode structures such as grammatical roles, parse trees, or game-board states.Probe models are trained to predict targets from embeddings, and representation modifications can test effects on outputs.
  • Computations: Circuit analysis searches for components implementing specific computations, including induction heads that predict repetitions from earlier sequence patterns.
  • Learning mechanisms: Reverse engineering can approximate transformer functions with simpler circuits, but ab initio analysis remains challenging and may require automated methods.
  • Learning mechanisms: Transformer parsing models can encode compact, interpretable structures through compression rather than requiring embedding dimensions proportional to grammar size times sentence length.The discussion notes that fairly good parsing can use about 20 nonterminals and that lower-dimensional projections can preserve useful structure.
  • Learning mechanisms: Once a circuit or computational model is identified, researchers can study how training data causes the model to learn that implementation, using theory, empirical analysis, and synthetic data.

8 Questions and discussion

The discussion considers what current LLMs may lack, including world models, planning, confidence judgments, and reflection, while stressing that strong conclusions remain premature. It also argues that understanding diverse capabilities requires connecting computational, mathematical, physical, psychological, and neuroscientific perspectives.

  • World models: Benchmark success does not establish that LLMs construct world models, while spatial-reasoning errors may indicate missing or ineffective non-linguistic representations.
  • World models: Evidence about LLM world models is unstable because later models may not reproduce earlier errors, making strong conclusions premature.
  • Computational perspectives: Constant-depth transformers are studied through circuit complexity, roughly associated with TC0, and autoregressive repetition turns the transition function into a finite-state-machine process.
  • Computational perspectives: LLM understanding draws on statistical physics, spin-glass theory, random matrices, geometry, and continuous mathematics alongside computer science.
  • Open capabilities: The notes identify planning, confidence judgments, and reflection as three capabilities that seem lacking in current LLMs.
  • Open capabilities: Because LLMs lack a dedicated search component and predict one word at a time without revising earlier predictions, planning is described as seriously handicapped.
  • Open capabilities: Hallucinated facts and citations make confidence judgments a major obstacle for practical applications, while next-word prediction alone may not provide epistemic status signals.
  • Open capabilities: LLMs are statistical models, so without additional non-statistical assumptions they cannot move from almost 100% confidence to 100% confidence.

A Grammars and parsing

Formal grammars represent language with production rules that generate strings and parse trees, while probabilistic and attribute extensions add uncertainty or values. The notes compare this symbolic parsing framework with LLMs, where learning and parsing are subsumed by model weights.

  • Formal grammars: A sentence diagram motivates formal parsing by representing clauses, grammatical roles, and the modifiers attached to words.
  • Formal grammars: Production rules rewrite a start symbol into grammatical strings, and recording the rewrites yields a parse tree.
  • Formal grammars: Context-free grammars restrict each rule’s left-hand side to one symbol, allowing nested structures that regular grammars cannot describe.
  • Extensions: Attribute grammars extend grammar rules so tokens such as VALUE can carry numerical or symbolic values, supporting arithmetic operations.
  • Limitations: CFGs are not really adequate for natural languages because natural-language structure contains inherent ambiguity and many special cases and exceptions.
  • Extensions: Probabilistic context-free grammars associate distributions with nonterminals and assign probabilities to generated strings, including zero probability for nongrammatical strings.
  • Parsing systems: A symbolic parsing system separates grammar learning from parsing, using one algorithm to learn a grammar and another to produce parse trees.
  • Parsing systems: LLM-style connectionist models subsume grammar learning and parsing in one model whose weights encode rules, and interpretability seeks to extract a parser and PCFG from it.
Loading 2307.05782v2…