PaperlayerView exact revision on arXiv

Source-linked AI summary

Language Models Use Trigonometry to Do Addition

Subhash Kantamneni, Max Tegmark

arXiv:2502.00873v1cs.AIcs.CLcs.LG

TL;DR

LLMs can perform simple arithmetic, but how they process addition remains poorly understood. The paper reverse engineers addition in three mid-sized LLMs by analyzing number representations and intervening on model components. It finds that the models represent numbers as generalized helices and manipulate them with a Clock-like procedure to produce answer logits.

  • Problem

    Limited understanding of how LLMs process even simple mathematical tasks motivates a representation-level study of LLM addition.

  • Method

    The paper reverse engineers three LLMs by modeling number representations, MLP and attention outputs, and neuron preactivations, then testing the account with causal interventions.

  • Results

    Three mid-sized LLMs represent numbers as generalized helices and manipulate them with the Clock algorithm to compute addition.

  • Takeaways & Limitations

    The work provides a representation-level explanation of an LLM mathematical capability by connecting number representations with the components that manipulate them.

  • Takeaways & Limitations

    The authors cannot isolate the exact mechanism that constructs helix(a + b), and the Clock algorithm may be one method among an ensemble that varies across models and tokenization schemes.

Abstract

from arXiv · show

Mathematical reasoning is an increasingly important indicator of large language model (LLM) capabilities, yet we lack understanding of how LLMs process even simple mathematical tasks. To address this, we reverse engineer how three mid-sized LLMs compute addition. We first discover that numbers are represented in these LLMs as a generalized helix, which is strongly causally implicated for the tasks of addition and subtraction, and is also causally relevant for integer division, multiplication, and modular arithmetic. We then propose that LLMs compute addition by manipulating this generalized helix using the "Clock" algorithm: to solve $a+b$, the helices for $a$ and $b$ are manipulated to produce the $a+b$ answer helix which is then read out to model logits. We model influential MLP outputs, attention head outputs, and even individual neuron preactivations with these helices and verify our understanding with causal interventions. By demonstrating that LLMs represent numbers on a helix and manipulate this helix to perform addition, we present the first representation-level explanation of an LLM's mathematical capability.

1. Introduction

The paper addresses limited understanding of how LLMs perform simple arithmetic by reverse engineering addition in three models. It identifies a helix-based Clock algorithm in which representations of addends are manipulated into an answer representation.

  • The study reverse engineers how GPT-J, Pythia-6.9B, and Llama3.1-8B compute a + b for a,b ∈[0,99].
  • The Clock algorithm models addition as rotating the a and b helices to create the a + b helix and read out the final answer.
  • All three LLMs represent a and b as helices and construct helix(a + b) on the last token, verified with causal interventions.
  • The work combines feature and circuit analysis by explaining both how numbers are represented and how downstream components manipulate those representations.

2. Related Work

Related work spans mechanistic interpretability of circuits and features, studies of circular or linear numerical representations, and competing explanations of arithmetic computation. This paper builds on those strands while targeting a representation-level account of LLM addition.

  • Mechanistic interpretability research studies either circuits, the components required for computations, or features, the representations of concepts in LLMs.
  • Prior work models numerical representations as circles or lines, while reporting limited causal evidence for those representations.
  • Research on modular addition introduced circular representations and the Clock algorithm, while the Pizza algorithm provides a rival explanation for some transformers.
  • Studies of LLM arithmetic identify circuits, neuron-based heuristics, and Fourier components as possible contributors to addition.

3. Problem Setup

The study evaluates autoregressive transformers on single-token addition problems and examines their hidden-state representations. The setup uses three mid-sized models, with GPT-J receiving the main analysis because its simple MLPs facilitate neuron interpretation.

  • The models are autoregressive transformers whose token hidden states combine multilayer perceptron and attention-layer outputs.
  • GPT-J and Pythia-6.9B use simple MLPs, whereas Llama3.1-8B uses a gated MLP; their tokenization ranges also differ.
  • On 10,000 addition problems with a,b ∈[0,99], GPT-J achieves 80.5% accuracy, Pythia-6.9B 77.2%, and Llama3.1-8B 98.0%.
  • GPT-J representations show sparse high-frequency Fourier components and a roughly linear first principal component over the studied range.

4. LLMs Represent Numbers as a Helix

The paper models numerical representations in LLMs as generalized helices combining periodic Fourier features with a linear component, and finds these structures causally support addition. Activation-patching analyses further connect the fitted helix to the variables used throughout the computation.

  • 4.1. Linear and Periodic Structure: GPT-J’s numerical representations show both linear structure over a ∈[0, 99] and periodic Fourier components with periods T = [2, 5, 10].The linear trend is visible in the first principal component, while Fourier analysis identifies high-frequency periodicity and pairwise distance and cosine-similarity periodicity.
  • 4.2. Parameterizing the Structure as a Helix: The proposed generalized helix combines one shared linear direction with k independent Fourier features, represented by 2k + 1 basis functions.The fitted periods are T = [2, 5, 10, 100], with the final period motivated by its magnitude and the base-10 number system.
  • 4.3. Fitting a Helix: The helix fit is obtained by projecting residual streams to 100 PCA dimensions, fitting Fourier bases by linear regression, and mapping the coefficients back to the full residual-stream space.The fitted helix subspace is visualized by projecting residual streams with the Moore-Penrose pseudo-inverse of the coefficient matrix.
  • 4.4. Evaluating the Quality of the Helical Fit: The helix remains causally relevant beyond trained values and is consistent with a nonlinear feature manifold, while its fitted Fourier components are mostly orthogonal and increase in magnitude with period.The reported continuity for untrained values provides additional evidence against the strongest form of the Linear Representation Hypothesis.
  • 4.3. Fitting a Helix: Activation patching shows that helical and circular fits preserve information used for addition, with the full k = 4 fits approaching layer-patching effects and outperforming the PCA baseline.The sharp performance increase between layer 0 input and layer 1 input is consistent with layer 0 processing being influential for numerical tasks.
  • 4.5. Is the Helix the Full Picture?: Ablating helix dimensions affects addition performance roughly as much as ablating the entire residual stream, although helical fits can underperform PCA on other numerical tasks.This supports the helix’s necessity for addition while leaving open the possibility that additional numerical structure is needed for other tasks.

5. LLMs Use the Clock Algorithm to Compute Addition

The paper argues that LLMs compute addition by representing numbers as helices and manipulating them with the Clock algorithm. Causal interventions and component-level analyses support a two-stage circuit in which MLPs construct helix(a + b), then downstream components read it out as the answer.

  • 5.1. Introducing the Clock Algorithm: The Clock algorithm represents a and b as helices, moves them to the last token, constructs helix(a + b), and reads the result into model logits.This pipeline is supported by analyses of attention heads, MLPs, and causal interventions.
  • 5.1. Introducing the Clock Algorithm: Last-token hidden states are well modeled by helix(a, b, a + b), and the a + b helix can fit them better than a 27-dimensional PCA using only 9 parameters.The causal influence of helix(a + b) supports its central role in the computation.
  • 5.2. Investigating Attention Heads: Activation and path patching identify MLPs as the main contributors to constructing the final answer, while attention heads mainly move or route helices.A sparse attention-head categorization also recovers nearly 80% of the effect of all 20 heads with four mixed heads.
  • 5.4.2. UNDERSTANDING MLP INPUTS: Top-neuron preactivations are periodic in a, b, and a + b, with common periods [2, 5, 10, 100], and fitted preactivations retain roughly 75% of actual-preactivation performance.These results support the view that individual neurons read from the helical representation.
  • 5.3. Looking at MLPs: The circuit uses MLPs 14-18 to build the a + b helix and MLPs 19-27 to read that helix and write the answer to model logits.The split is supported by both output modeling and direct-effect measurements.
  • 5.5. Limitations of Our Understanding: The authors conclude that LLMs use the Clock algorithm for addition, while noting that the exact mechanism for constructing helix(a + b) remains unresolved and may vary across models.They hypothesize trigonometric identities but cannot isolate that computation, and note possible alternative or ensemble implementations.

6. Conclusion

The study finds that three mid-sized LLMs represent numbers as generalized helices and manipulate them with the Clock algorithm to compute addition. This mechanism offers a representation-level account of mathematical computation and motivates further investigation of LLM algorithms and errors.

  • Three mid-sized LLMs represent numbers as generalized helices and manipulate them with the Clock algorithm to compute addition.
  • The Clock algorithm provides evidence that LLMs trained on general text can naturally learn complex mathematical algorithms.
  • The authors argue that understanding these algorithms matters for safe AI and for explaining model errors.
  • The study tests addition prompts across all 10,000 pairs a, b ∈[0, 99] using GPT-J, Pythia-6.9B, and Llama3.1-8B.

B. Additional Results on the Structure of Numbers

Additional analyses support a structured, nonlinear representation of numbers in GPT-J. The representation combines local linearity with periodic Fourier features and continuity consistent with a nonlinear manifold.

  • GPT-J representations show periodicity, but distances grow sublinearly across two-digit numbers and locally linearly for a ∈[0, 9].
  • The first principal component is linear until a discontinuity at a = 100, suggesting that three-digit numbers occupy a different representational space.
  • The helical Fourier-feature magnitudes roughly increase with period, matching the ordering found in the Fourier decomposition.
  • Helical components are mostly orthogonal, except notably for the T = 100 sine component and the linear component.
  • Projected held-out numbers fall near their expected positions, supporting the interpretation of the helix as a continuous nonlinear feature manifold.

C.2. Additional Causal Experiments for Helix Fits

Causal experiments show that helix fits capture addition-relevant computation across models and generalize beyond the fitted values. They also identify influential attention heads and downstream pathways involved in constructing and using helix(a + b).

  • Helix and circular fits outperform the PCA baseline on held-out values, while randomized helices are not causally relevant.
  • Ablating helix dimensions from the residual stream is roughly as destructive as ablating the entire layer, supporting their causal necessity for addition.
  • On other numerical tasks, helix fits are effective but sometimes underperform PCA baselines, indicating that additional structure may be required.
  • Helix(a + b) fits last-token hidden states in Pythia-6.9B and Llama3.1-8B, but the weaker Llama3.1-8B fit may indicate additional algorithms.
  • For attention heads, four mixed heads recover almost 80% of the effect obtained from patching all 20 influential heads.
  • Path patching indicates that a, b and mixed heads mainly affect downstream MLPs, whereas a + b heads rely predominantly on upstream MLPs.

D.2. MLPs and Neurons

MLP and neuron analyses locate a sparse computational circuit whose activity follows the helix structure. Most influential neurons concentrate in identified circuit MLPs and encode the helix’s characteristic periods.

  • Patching 11 MLPs achieves 95% of the effect of patching all MLPs.
  • Roughly 1% of neurons are sufficient to achieve an 80% success rate when other neurons are mean-ablated.
  • Almost 75% of top neurons are located in the 11 circuit MLPs identified by the analysis.
  • Roughly 700 neurons are required to achieve 80% of the direct effect from patching all 4,587 top neurons.
  • The weaker helical fit for Llama3.1-8B may reflect non-Clock algorithms associated with its gated MLPs.
  • Fourier decomposition of top-neuron preactivations reveals spikes at T = [2, 5, 10, 100], matching the periods used in the helix parameterization.

E. Why Use the Clock Algorithm at All?

The paper argues that the Clock algorithm’s helical representation provides redundancy that linear representations lack, helping models perform addition despite imperfect precision. Experiments show that linear addition performs poorly even with an excellent geometric fit, whereas GPT-J remains substantially more accurate.

  • Patching k = 20 attention heads restores more than 80% of the behavior obtained by patching all k = 448 heads.
  • Attention-head outputs are well modeled by helix(a, b, a + b), with a,b heads earlier, mixed heads in middle layers, and a + b heads later.
  • Only k = 11 MLPs are required to achieve 95% of the effect, while roughly 700 neurons recover 80% of the direct effect of all k = 4587 high-impact neurons.
  • Top neuron preactivations are periodic in a + b, with prominent periods T = [2, 5, 10, 100], while more impactful neurons are generally fit better by helix-inspired models.
  • Linear addition achieves less than 20% accuracy below α = 100, compared with more than 80% for GPT-J.The linear representation was fit with R2 = 0.997 before testing addition.
  • The authors interpret modular circles in the Clock algorithm as built-in redundancy that mitigates errors from imperfect representations.

F. Investigating Model Errors

The paper examines GPT-J’s addition errors and finds that most incorrect numeric answers differ from the correct answer by ±10. Analysis rejects incorrect carrying as the explanation and links the error pattern to periodicity in the model’s logits.

  • 45.7% of incorrect numeric answers are off by −10 and 27.9% are off by 10, together exceeding 70%.
  • A chi-squared test finds that unit digits summing to more than 10 are not more likely for −10 errors, falsifying the carrying hypothesis.
  • Approximately k = 693 highest-direct-effect neurons recover 80% of total direct effect, and their LogitLens contributions typically boost and suppress answer tokens periodically.
  • Fourier decomposition of final logits identifies 10 as the most common top period, making ±10 errors consistent with the model’s answer-space periodicity.

G. Tooling and Compute

The intervention experiments used nnsight and were run on a single NVIDIA RTX A6000 GPU with 48GB of VRAM. Under this configuration, the experiments were reported as reproducible in two days.

  • The experiments used the Python library nnsight for intervention experiments on language models.
  • All experiments ran on a single NVIDIA RTX A6000 GPU with 48GB of VRAM.
  • The reported configuration allows all experiments to be reproduced in two days.
Loading 2502.00873v1…