PaperlayerView exact revision on arXiv

Source-linked AI summary

The Geometry of Metaphor: Probing and Steering Metaphor Representations in Large Language Models

Shihui Li, Mette Andersen, Martha Lewis

arXiv:2610.31520v1cs.CLcs.AI

TL;DR

The paper investigates how metaphor is represented in LLMs and whether its metaphoricity can be controlled. Using activation-based methods to identify metaphor-related subspaces, it reports steering effects in classification and generation, while noting limitations from the simplicity of the metaphor datasets.

  • Problem

    The paper asks whether metaphor has an identifiable representation in LLMs and whether that representation can be used to steer output.

    Source: §1 · p. 1

  • Method

    The authors use activation patching and distributed alignment search to identify metaphor-related directions or subspaces and intervene on model activations.

    Source: §1 · p. 1, §2 · p. 2, §4.1.2 · p. 4

  • Results

    The authors report metaphor-related directions across models and datasets, causal steering effects on classification outputs, and generation steering toward more metaphorical text.

    Source: §6 · pp. 7–8

  • Takeaways & Limitations

    Activation interventions offer a way to control generated metaphoricity, with the paper presenting this as a potential alternative to prompting.

    Source: §6.1 · p. 9

    The controlled datasets use simplistic metaphor examples, while metaphor in real discourse can extend across sentences and paragraphs.

    Source: §7 · p. 9

Abstract

from arXiv · show

Metaphor is a crucial aspect of human cognition. As such, we would like to understand how it is represented in large language models (LLMs) and how it might be controlled. We use mechanistic interpretability techniques to identify causal subspaces in a range of LLMs covering different architectures and sizes. We show that we are able to steer output on classification tasks across different parts of speech. We are also able to steer output on generation tasks, indicating that these techniques can be used to control the metaphoricity of LLM output.

1 Introduction

The paper asks how metaphor is represented in LLMs and whether its metaphoricity can be steered, addressing a mechanistic interpretability gap. It uses activation patching and distributed alignment search to identify metaphor-related directions and tests steering across models and tasks.

1 Introduction

  • The work targets a mechanistic-level account of metaphor representation, beyond earlier investigation of metaphor in models.The introduction notes prior work on metaphor but says it had not yet examined the issue at the mechanistic level.Source: §1 · p. 1
  • The authors identify metaphor-related directions and subspaces in several models and report causal effects on classification and generation outputs.The generation intervention steers output toward more metaphorical text.Source: §1 · p. 1
  • Activation patching and distributed alignment search are used to find a metaphoricity direction or subspace within a model.Source: §1 · p. 1
  • Metaphor directions are somewhat consistent across different datasets.Source: §1 · p. 1
  • The evaluation covers seven models from four families, spanning small 7–9B and medium-large 27–70B sizes.Source: §1 · p. 1

2 Background and Related work

The section defines metaphor identification and interpretation as distinct tasks and situates the paper among work on model representations and output control. It introduces activation patching and distributed alignment search as approaches relevant to finding and steering metaphor representations.

Metaphor identification and interpretation

  • Metaphor identification asks whether a word is metaphorical, while metaphor interpretation asks for the meaning of a metaphorical utterance.Source: §2 · p. 2

Activation Patching

  • Activation patching saves activations from one input and substitutes them into another to test which layers and tokens affect output.Source: §2 · p. 2

Distributed Alignment Search

  • Distributed Alignment Search seeks causal concepts in activations under the assumption that full-vector activations may be polysemantic.The paper uses DAS to find a metaphoricity concept in the residual stream and compares its steering with mass-mean steering.Source: §2 · p. 2

Controlling Generation

  • Prior generation-control approaches include separate models, prompt-based style control, and inference-time interventions for truthfulness.Source: §2 · p. 2

Summary

  • The paper investigates metaphor separability, steering toward literal or metaphorical output, and whether steering vectors generalize across datasets.It frames this as activation-based control rather than prompting or building an external model.Source: §2 · p. 2

3 Datasets and Tasks

The study uses controlled noun and adjective datasets, with short, syntactically parallel sentences and a single metaphorical target, to support probing. It evaluates metaphor identification, paraphrase interpretation, and meaning generation using these datasets.

3.1 Datasets

  • The datasets use short, structurally parallel sentences with one metaphorical target; verb data was excluded because model performance was insufficient for interpretability analysis.Source: §3.1 · p. 2
  • NOUN contains 130 manually verified sentence tuples, rewritten from creative metaphors into “The noun1 is (a/an) noun2” form while preserving their metaphorical mappings.The source examples came from Fig-QA; idioms, multi-clause descriptions, and long narratives were removed before rewriting.Source: §3.1 · pp. 2–3
  • ADJ’s test set contains 100 adjective–noun pairs with literal and metaphorical usages; DOM spans eight adjective domains and is evaluated on 100 held-out examples per steering direction.Source: §3.1 · p. 3

3.2 Tasks

  • Tasks comprise four-shot literal/metaphor identification, apt/inapt paraphrase interpretation on NOUN, and five-shot meaning generation from adjective sentences in DOM.Source: §3.2 · p. 3

4 Methods

The methods use activation patching and distributed alignment search (DAS) to probe metaphor representations, then use mass-mean and DAS interventions to steer classification and generation. DAS identifies a linear subspace, while generation steering intervenes at each decoded token.

4.1 Identifying areas encoding metaphoricity

  • Activation patching probes identification and interpretation across NOUN, ADJ, and DOM; DAS identifies a linear metaphoricity subspace for DOM and generation.Source: §4.1 · p. 4

4.1.1 Activation Patching

  • For paired literal and metaphor prompts, activation patching swaps hidden states across token positions and layers, then measures the change in log probability between output labels.Source: §4.1.1 · p. 4

4.1.2 Distributed Alignment Search

  • DAS trains an orthonormal, k-dimensional subspace at the causally implicated final-period token, optimizing interchange intervention accuracy (IIA).The intervention replaces the base activation’s subspace component with the source activation’s component.Source: §4.1.2 · p. 4
  • DAS training alternates literal-to-metaphor and metaphor-to-literal interventions for 1,000 Adam steps with model weights frozen; IIA measures the fraction correctly flipped.Source: §4.1.2 · p. 4

4.2 Model steering

  • For initially incorrect predictions, steering uses full-vector mass-mean vectors or means of metaphor and literal activations in the DAS subspace, in either direction.Means are computed from correctly predicted samples; on DOM, DAS is trained on the full dataset and separately by domain.Source: §4.2 · p. 5

4.2.1 Generation of multiple tokens

  • Generation steering patches every decoded token; baseline, mass-mean, and DAS outputs are compared across four α values and both directions, with first sentences judged after degeneracy filtering.Flip rate counts target-matching judge labels, treating degenerate completions as zero and considering samples judged correctly at baseline.Source: §4.2.1 · p. 5

4.3 Models Evaluated

  • Evaluation covers seven instruction-tuned models across four model families, tested on metaphor identification and interpretation; model sizes range from 7B to 72B parameters.Source: §4.3 · p. 5

5 Results

The results examine baseline performance, locate and compare metaphor-related activations, and test whether interventions can steer classification and generation. Across datasets, activations show linear separation, and global DAS matrices outperform domain-specific matrices on held-out evaluation.

5.1 Baseline Performance

  • Across identification and interpretation tasks, models perform reasonably well except Gemma 27B on DOM, which is excluded from further analysis on that dataset.Non-instruction-tuned models were also excluded from interpretability analyses because their task performance was not strong enough.Source: §5.1 · p. 5, §5.2.1 · p. 5, Table 2 · p. 6

5.2.2 DAS training

  • Global DAS matrices have higher IIA than domain-specific matrices on held-out evaluation sets.The comparison addresses whether metaphor is encoded across domains or in domain-specific subspaces.Source: Table 4 · p. 6, §5.2.2 · p. 6

5.2.1 Activation patching

  • Literal and metaphorical prompts implicate several activation groups, while the first two principal components separate metaphorical from literal and apt from inapt examples.Group (b), comprising the final period and preceding token, is hypothesized to encode metaphoricity.Source: §5.2.1 · pp. 5–6, Figure 1 · p. 6, Figure 2 · p. 7

5.3.1 Identification and Interpretation

  • Mass-mean and logistic-regression probes are compared using normalized indirect effects to assess steering on identification and interpretation tasks.The reported Llama-3.1-8B results are in Table 5.Source: Table 5 · p. 7, §5.3.1 · p. 7

5.3.2 Mass-mean and DAS steering

  • On initially incorrect DOM identification items, mass-mean and DAS steering achieve fairly high repair performance across models.The table reports the proportion corrected by full-residual-stream patching and by intervention on the learned subspace.Source: Table 6 · p. 7, §5.3.2 · p. 7

5.3.3 Generation

  • Across models, α = 5 enables the most effective generation steering on average.The paper also qualitatively inspects generated outputs, including successful and unsuccessful examples.Source: §5.3.3 · p. 7

6 Discussion and Conclusions

The discussion reports interpretable metaphoricity directions across models and datasets, with interventions affecting classification and generation. It also identifies limits in intervention strength and in the controlled settings used.

6 Discussion and Conclusions

  • DAS matrices alter classification outputs; global subspaces transfer better than domain-specific ones, and mass-mean steering outperforms logistic-regression steering.The discussion describes these directions as causally effective across models and datasets.Source: §6 · pp. 7–8
  • Steering from literal to metaphorical output yields higher NIEs than steering in the reverse direction.The authors suggest that creating a literal meaning from a metaphorical one may be conceptually harder.Source: §6 · p. 8
  • Mass-mean probes are generally more effective than DAS for generation steering, although effectiveness differs across models.Table 7 reports flip rates at α = 5; Table 8 provides LLaMA-8B example generations at that steering strength.Source: §6 · p. 8, Table 7 · p. 8, Table 8 · p. 8
  • IIA scores are moderate and variable, indicating that some metaphoricity information lies outside the identified concept.The authors note that perfect IIA may not be achievable in practice with DAS concept-finding.Source: §6 · p. 8
  • For Gemma 2 9B, a matched-n global DAS matrix achieved better or equal transfer IIA in 6/8 domains, addressing training-size confounding.The matched-n matrix used n = 255 samples, matching the smallest training domain.Source: §6 · p. 8
  • Generation steering has substantially lower IIA than classification steering and requires scaling the signal by α = 5.Human annotation of 100 Gemma 9B outputs had raw agreement 0.79 and Cohen’s kappa 0.60 with the LLM judge.Source: §6 · pp. 8–9
  • The experiments use controlled, templated prompts to localize causal effects, so the findings do not test more naturalistic contexts such as poems.The authors identify this choice as limiting external validity.Source: §6 · p. 9

6.1 Conclusions

  • The conclusions characterize metaphoricity as an interpretable direction across LLMs and report causal output changes in classification and generation.The authors suggest these techniques may offer greater control over style modification than prompting.Source: §6.1 · p. 9

7 Limitations and Risks

The paper bounds its findings to English metaphor and controlled, relatively simple settings, and notes constraints on model access and possible steering risks. The accompanying prompts and plots illustrate the tested representations and tasks.

7 Limitations and Risks

  • The work covers English metaphor only; linear separability and interpretable directions may differ in other languages.Source: §7 · p. 9
  • The datasets use simplistic metaphor examples, while metaphor spanning multiple sentences, paragraphs, or discourse remains for future research.Source: §7 · p. 9
  • Experiments omit the most highly performing closed models because they lack open weights and computational resources would preclude testing them.Source: §7 · p. 9
  • The authors assess current risks as fairly low but note that steering could target undesirable as well as desirable behavior.Source: §7 · p. 9

A NOUN identification prompt

B Activation patching plots

  • The activation-patching plots show adjective and noun identification, plus noun interpretation, by patching activations between task prompts.Source: Figure 3 · p. 12, Figure 4 · p. 12

C PCA plots

D Domain-specific DAS training at full-stop token position

The appendix captions describe domain-specific DAS IIA across models and transfer IIA for global DAS on held-out domains.

D Domain-specific DAS training at full-stop token position

  • Table 9 reports domain-specific DAS IIA as mean ± standard deviation across models, with the best result per domain marked.Source: Table 9 · p. 15

E Transfer: validation IIA using global DAS instead of domain-specific DAS

  • Table 10 reports mean transfer IIA with 95% confidence intervals for global DAS evaluated on held-out domains.Source: Table 10 · p. 15

F Steering incorrect predictions

The table caption defines a comparison of mass-mean steering and DAS intervention on initially incorrect items.

F Steering incorrect predictions

  • Table 11 reports flip rates on initially wrong items for mass-mean steering versus DAS intervention, with means and 95% confidence intervals.Source: Table 11 · p. 16

G LLM as a judge

The appendix reports generation-steering evaluation criteria, a matched-sample control, qualitative examples, and the calculation and interpretation of normalized indirect effects (NIEs).

G LLM as a judge

  • Generation flip rates are reported by steering scale and method for both literal-to-metaphor and metaphor-to-literal directions, with degenerate outputs counted as failed flips.The literal-to-metaphor analysis excludes few-shot prompts and includes only pairs whose baseline explanation the judge classifies correctly; the reverse-direction table uses baseline-correct items and also excludes few-shot prompts.Source: Table 12 · p. 16, Table 13 · p. 16

H Example generations

  • The qualitative examples mark whether the judge’s label flipped to the steered direction for mass-mean and DAS interventions.A check mark indicates a flip; a cross indicates no flip.Source: Table 14 · p. 17

I Matched-n control

  • For Gemma 2 9B, 6/8 matched-n cells achieved a comparable transfer IIA score to the unfiltered global DAS matrix.The matched domain-specific training size was n = 255.Source: Table 15 · p. 17

J.1 NIE calculation

  • NIE compares class-specific average metaphorical-versus-literal probability differences before and after intervention; 0 means no causal effect and 1 a complete prediction shift.The differences are averaged over metaphoric and literal examples, respectively.Source: §J.1 · p. 18

J.2 Full NIE results

The appendix provides full NIE and flip-rate tables alongside generation-prompt and judge details, dataset sources, computational settings, and AI-assistant use.

J.2 Full NIE results

  • Tables report mass-mean difference and logistic-regression probe NIEs for identification across eight models and interpretation on the Noun dataset.Higher NIE indicates stronger causal steering effects.Source: Table 16 · p. 19, Table 17 · p. 19

J.3 Flip rate results

  • The flip-rate table covers mass-mean probes on identification and interpretation, steering correctly classified test examples toward the opposite label.It defines before and after as counts retaining the original label, and flip rate as the proportion of originally correct predictions flipped.Source: Table 18 · p. 20

J.4 Prompt format

  • Generation uses a plain-continuation meaning prompt with five hardcoded few-shot examples, and the hook patches the last-token hidden state at each generation step.No chat template is applied; the patched position matches the one used for DAS training.Source: §J.4 · p. 21

K Prompt for LLM judge

  • The judge prompt asks whether an adjective meaning description conveys a literal or metaphorical sense and requires a one-word label.It distinguishes concrete or sensory properties from abstract, figurative, or emotional properties.Source: K Prompt for LLM judge · p. 21

L Dataset Usage

  • The study uses metaphor-research datasets from Liu et al. (2022b), Gutiérrez et al. (2016), and Tsvetkov et al. (2014), with slight format changes.The passages state that the datasets are used for their original metaphor-research purpose and are freely available in the public domain.Source: L Dataset Usage · p. 21

M Computational resources

  • Experiments took about 125 hours on an H100 GPU; checked settings included generation steering strengths α ∈ {1, 2, 5, 10} and DAS dimensions k ∈ {4, 8, 16, 32}.Source: M Computational resources · p. 21

N Use of AI assistants

  • AI assistants were used for code debugging, generating the generation-experiment prompt, and generating tables, with numbers double-checked manually.Source: N Use of AI assistants · p. 21

Related papers

Loading 2610.31520v1…