PaperlayerView exact revision on arXiv

Source-linked AI summary

Bridging Linguistic Structure and Mechanistic Interpretability for Conceptual Interpretation in Language Models

Nura Aljaafari, Danilo S. Carvalho, André Freitas

arXiv:2408.11827v2cs.CL

TL;DR

Mechanistic studies have largely focused on factual retrieval, leaving unclear whether transformer components support compositional interpretation of definitions. DSRA applies causal tracing to reverse-dictionary prediction and overlays definitional semantic roles; it finds structured restoration patterns across components and languages, with DSR categories represented among high-impact states.

  • Problem

    Mechanistic findings derived predominantly from factual retrieval do not establish whether or how transformer components support compositional interpretation of definitional structure.

    Source: §1 · p. 1

  • Method

    DSRA applies causal tracing to reverse-dictionary prediction and groups restoration traces by definitional semantic roles to examine alignment with definitional functions.

    Source: §1 · pp. 1–2

  • Results

    Across English and Spanish, localisation patterns are consistent; differentia_quality accounts for over 31% of top-50 MLP restoration states, while supertype and differentia_event each contribute approximately 6%.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The DSR overlay provides an interpretive lens connecting activation patching with formal lexical semantics and reveals regularities beyond token identity.

    Source: §1 · p. 2, §6 · p. 9

    The study uses one scale of autoregressive transformer and sentence-level meaning with single-token prediction, which does not fully reflect conceptual processing across other tasks.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Understanding how language models compose meaning from linguistic input remains a central problem in interpretability research. Mechanistic studies have attributed functional roles to core transformer components; however, these findings derive largely from factual retrieval settings. Whether the same mechanisms support \textit{conceptual interpretation}, the compositional mapping from definitional expressions to abstract meaning, remains insufficiently characterised. We introduce \textit{DSRA} (Definitional Semantic Role Analysis), a methodology that applies causal tracing within the reverse dictionary task and augments restoration traces with definitional semantic roles (DSRs) grounded in Argument Structure Theory. This linguistic overlay identifies which compositional functions (e.g., genus, differentia quality) are associated with high-recovery states, extending activation patching beyond token-level localisation. Applied to GPT-J-6B (English) and BERTIN GPT-J-6B (Spanish), the results show that MLP layers associate content-bearing tokens with high-specificity DSR categories in early layers, MHA layers distribute integration across middle-to-upper layers with concentration at the final token, and hidden states aggregate information in upper layers. Alignment between restored states and DSR categories indicates systematic correspondence between internal activations and definitional structure, with consistent localisation patterns across both languages.

1 Introduction

Mechanistic studies have largely examined factual retrieval, leaving unclear whether the same component behaviours support compositional conceptual interpretation. DSRA applies causal tracing to reverse-dictionary definitions and overlays semantic roles; its analyses identify recurring patterns across components and languages.

1 Introduction

  • Prior mechanistic findings derive predominantly from factual retrieval prompts and do not determine how components support compositional interpretation of definitional structure.Conceptual interpretation depends on integrating definitional structure, thematic roles, and predicate–argument relations.Source: §1 · p. 1
  • DSRA links causal tracing to formal lexical semantics in the reverse dictionary task, where a model predicts the term specified by a natural-language definition.The method uses definitional semantic roles (DSRs) to interpret restoration traces.Source: §1 · pp. 1–2
  • The DSR overlay groups definition spans by compositional function, allowing restoration traces to be assessed beyond token identity.The taxonomy includes supertype, differentia_quality, and differentia_event.Source: §1 · p. 2
  • 31% of top-50 MLP restoration states are differentia_quality, while supertype and differentia_event each contribute approximately 6%.The integrated analysis reports consistent localisation patterns across English and Spanish.Source: §1 · p. 2

2 Background

The background frames dictionary definitions as structured expressions whose parts compose the meaning of a defined term. It motivates analysing whether internal model contributions correspond to linguistic components such as categories and distinguishing properties.

2 Background

  • Intensional definitions specify meaning through conditions on term usage and commonly follow a genus–differentia structure.The genus gives a broader category, which the differentia restricts through distinguishing properties.Source: §2 · pp. 2–3
  • In the example “service,” “work” is the genus and the remaining clause is the differentia.This structure is presented as a controlled basis for studying how models integrate meaning.Source: §2 · p. 3
  • Under the paper’s Montagovian framing, a complex expression’s meaning depends on its parts and how they are combined.Argument Structure Theory models a definiens as predicate–argument structures with thematic roles.Source: §2 · p. 3
  • The formalisation binds lexical content to thematic roles and composes lexical-semantic units into definition-level meaning.The paper equates the meanings of the definiendum and definiens.Source: §2 · p. 3
  • In the reverse dictionary task, a generative model receives a definition and predicts the defined term as its next token.The task provides a correctness criterion for the interpretation.Source: §2 · p. 3
  • Knowledge-localisation and sparse-autoencoder findings motivate testing whether related tools can characterise compositional interpretation beyond factual retrieval.The cited work includes interventions on MLP representations and analyses of approximately monosemantic features.Source: §2 · p. 3

3 Methodology

The methodology adapts causal tracing to definition-to-term prediction, then interprets restoration scores through linguistic categories. It evaluates the approach on English and Spanish WordNet-derived datasets with models sharing a base architecture.

3.1 Localisation via Causal Tracing

  • Causal tracing compares clean, corrupted, and patched executions to identify internal states whose restoration recovers the definiendum prediction.The corrupted run adds Gaussian noise to definition-token embeddings; the patched run restores selected states to clean-run values.Source: §3.1 · p. 3, Figure 2 · p. 4
  • Restoring individual component states can establish causal sufficiency for prediction, but does not characterise the full causal graph of component interactions.The method targets the correct definiendum token at the final position.Source: §3.1 · pp. 3–4

3.2 Recovery Metric and Semantic Alignment

  • The component-specific recovery score measures the fraction of corruption-induced degradation recovered by restoring a state.Scores are clipped to [0, 1] and averaged across 10 independent corrupted executions per input.Source: §3.2 · p. 4
  • MLP and MHA restorations span contiguous 10-layer windows, whereas hidden-state restorations target individual layer–token pairs.Top-K states are grouped by token position, DSR, and part of speech; Spanish uses PoS only because DSR labels are unavailable.Source: §3.2 · pp. 4–5

3.3 Datasets and Experimental Setup

  • The reverse dictionary task combines multi-part definitions into one prediction, providing a verifiable target for per-example causal tracing.The setup acknowledges that some high-recovery states may reflect lexical co-occurrence rather than structured composition.Source: §3.3 · p. 5
  • The study uses WordNet-derived English and Spanish datasets, restricting defined terms to single tokens for unambiguous prediction correctness.English WordNet has 8,348 samples and Spanish WordNet has 7,815; DSR annotations are available only for English.Source: §3.3 · p. 5
  • GPT-J-6B is used for English and BERTIN GPT-J-6B for Spanish, with a shared base architecture enabling comparison across languages.The models are fine-tuned using LoRA and QLoRA, respectively.Source: §3.3 · p. 5

4 Empirical Analysis

The empirical analysis uses top-K restoration states and compares localization across components, definitions, and languages. It finds distinct component patterns, alignment with definitional roles, and localization stability under paraphrase and across English and Spanish.

4.1 MLP: Content Association in Early and Upper Layers

  • MLP recovery is bimodal, with early content-token and upper-layer/final-token regimes when lexical cues are weak.The early-layer pattern appears at content-bearing tokens; weaker lexical cues shift recovery toward upper layers and the final token.Source: §4.1 · pp. 5–6, Figure 3 · p. 6
  • Over 31% of top-50 MLP restoration states are differentia_quality, versus approximately 6.7% differentia_event and 6.2% supertype.The authors report that differentia_quality is more concentrated than expected under a frequency-matched null model.Source: Table 1 · p. 6, §4.1 · p. 6

4.2 MHA: Distributed Integration in Middle-to-Upper Layers

  • MHA recovery centres on the final token across layers 12–25, while top-10 states are prompt-dominated and top-50 include content-bearing roles.Its layer distribution is unimodal and negatively skewed, unlike MLP's bimodal pattern.Source: §4.2 · p. 6

4.3 Hidden States: Upper-Layer Concentration

  • Hidden-state recovery concentrates at the final token in layers 20–27, more strongly in upper layers than MHA.Differentia_quality (32%) and differentia_event (7%) remain among top-50 restored states, indicating definitional information persists into the final representation.Source: §4.3 · pp. 6–7, Table 1 · p. 6, Figure 5 · p. 7

4.4 Validation and Cross-Lingual Stability

  • Paraphrases preserve broad localisation but shift high-impact positions; Spanish repeats localisation patterns, while DSR semantic alignment is evaluated only in English.The Spanish MLP-position differences are consistent with tokenisation variation.Source: §4.4 · p. 7

4.5 Synthesis: A Coarse-Grained Account

  • Component patterns oppose uniform localisation: MLP is bimodal, MHA and hidden states are unimodal, and DSR grouping overrepresents differentia_quality relative to span frequency.This correspondence does not establish a complete account of compositional computation; testing the coarse-grained account with finer causal interventions is left to future work.Source: §4.5 · pp. 7–8, Figure 6 · p. 8

5 Related Work

Related work identifies functional roles and linguistic information in transformer components, but leaves open how definitional structure is composed during conceptual interpretation. DSRA brings causal intervention and semantic-role analysis to that question.

Mechanistic interpretability

  • Probing studies identify linguistic knowledge in transformer layers (Tenney et al., 2019), while SAEs isolate interpretable features (Shu et al., 2025), but neither directly characterises composition across components.The stated gap concerns how information is composed and integrated during meaning construction.Source: §5 · p. 8

Activation patching and causal tracing

  • ROME-style causal tracing directly intervenes on internal states (Meng et al., 2022), whereas gradient-based attribution estimates contributions; DSRA adds semantic-role labels to restoration traces.This extends localisation from token positions to semantic roles.Source: §5 · p. 8

Definition modelling and reverse dictionary

  • DSRA jointly examines MLP, MHA, and hidden states and relates recovery to span-level definitional structure, addressing work on reverse-dictionary inference based on probing rather than intervention (Xu et al., 2024).The paper situates this approach at the intersection of mechanistic interpretability, causal analysis, and DSR-based definition analysis.Source: §5 · p. 9

6 Conclusion

The conclusion presents DSRA as a way to combine causal tracing with definitional semantic roles, then reports component-specific localization patterns and semantic alignment across English and Spanish models. It identifies finer-grained tracing, SAE integration, and broader tasks as future directions.

6 Conclusion

  • DSRA augments causal tracing in the reverse dictionary task with definitional semantic roles to connect activation patching and formal lexical semantics.The study focuses on localization and semantic alignment; causal intervention for model editing is left to future work.Source: §6 · p. 9
  • MLP states cluster bimodally in early and upper layers, MHA states concentrate in middle-to-upper layers, and hidden states concentrate in upper layers at the final token across English and Spanish.The DSR grouping also finds content-bearing roles, especially differentia_quality, among high-impact states.Source: §6 · p. 9
  • Future work may trace individual attention heads, integrate SAEs for feature-level DSR alignment, and test the framework beyond definitional interpretation.Source: §6 · p. 9

Limitations

The study’s scope and data constrain how broadly its localization and alignment findings can be interpreted. Its WordNet-based definitions and annotations may not reflect freer text or cross-linguistically stable semantic categories.

Limitations

  • The study uses one scale of autoregressive transformer and single-token prediction from sentence-level meanings, which does not capture conceptual processing across other tasks.Source: Limitations · p. 9
  • The analysis treats component types individually, studies MHA as a whole, and provides DSR annotations only for English.Source: Limitations · p. 9
  • WordNet senses may be overly fine-grained and its hierarchies may not align with cross-linguistically stable semantic categories, affecting annotation quality and generalisability.Source: Limitations · p. 10
  • Whether the observed alignment persists with freer explanatory-dictionary or corpus-derived definitions remains open.Source: Limitations · p. 10

Ethical Considerations

The ethical considerations identify risks associated with the dataset, annotation, and potential use of the findings, while noting that code and datasets are released for replication and verification.

Ethical Considerations

A Dataset Creation, Filtering, and Prompt Selection

The appendix describes dataset eligibility and construction, prompts and filtering, trace normalisation and annotation, alternative definitions, and causal tracing and model choices. These decisions support single-token reverse dictionary analysis and cross-lingual comparison.

A Dataset Creation, Filtering, and Prompt Selection; A.1 English WordNet (EWN)

  • Dataset/model choices required compositional semantic processing, verifiable targets, structured definitional analysis, and per-example causal-tracing compatibility; retained samples required single-token targets (8,348 examples; 80/20 training/testing).Definitions were limited to 25 words, and samples were restricted to alphabetical characters.Source: A Dataset Creation, Filtering, and… · p. 12, §A.1 · p. 12

A.2 Spanish WordNet (SWN)

  • Spanish WordNet construction uses available Spanish entries or translated English WordNet definitions, yielding 7,815 examples with a 30/70 train–test split.Definitions are limited to 25 words, covering 99% of samples; the split maximises held-out definitions while retaining training data for stable task performance.Source: §A.2 · p. 12

A.3 Preprocessing and Prompt Selection

  • Three prompt formats were tested, and simple prompts without added metadata gave the highest accuracy; tracing includes only examples correctly predicted under clean conditions.Source: §A.3 · pp. 12–13

A.4 Length Normalisation

  • Trace token positions are resampled by linear interpolation within the 10th–90th percentile definition-length band, and examples outside it are excluded from length-normalised aggregation.Source: §A.4 · p. 13

A.5 Definitional Semantic Role Annotation

  • DSR labels follow Silva et al. (2016), but annotations are available only for English WordNet definitions; Spanish definitions are analysed without DSR alignment.Source: §A.5 · p. 13

A.6 Alternative Definitions

  • Five GPT-3.5 alternative definitions are manually filtered for semantic similarity, plausibility, and absence of the definiendum; only correctly predicted alternatives enter tracing.Source: §A.6 · p. 13

B.1 Causal Tracing Specification

  • Causal tracing corrupts definition-span embeddings with independent Gaussian noise, then restores component activations across a contiguous 10-layer window or a hidden state at one layer-token pair.Table 2 lists DSR roles and definitions following Silva et al. (2016).Source: §B.1 · p. 13, Table 2 · p. 13

B.2 Models and Selection Rationale

  • GPT-J-6B and BERTIN GPT-J-6B share a base architecture for English–Spanish comparison, and both are fine-tuned for reverse dictionary prediction using LoRA.Model selection also prioritises activation access for tracing and computational feasibility.Source: §B.2 · p. 14

B.3 Experimental Setup

The setup adapts causal tracing to reverse-dictionary inference, then uses DSR labels and robustness analyses to examine how components localise and integrate definitional information. Across the reported checks, localisation patterns persist across part of speech, definition length, and alternative definitions.

B.3 Experimental Setup

  • Experiments used an NVIDIA RTX A6000 GPU and report architecture and LoRA fine-tuning hyperparameters in Tables 3 and 4.The supplied table captions identify the models and the reverse-dictionary task but do not specify hyperparameter values.Source: §B.3 · p. 14, Table 3 · p. 14, Table 4 · p. 14

C.1 Transformer Formalisation

  • The formalisation describes a decoder-only transformer whose hidden states are updated through a residual stream, with attention, MLP, and a final-position prediction head.MHA concatenates outputs from independent attention heads, which attend over causally permitted positions.Source: §C.1 · pp. 14–15

C.2 ROME Formalisation

  • The method adapts ROME’s causal localisation procedure to the reverse-dictionary task and does not perform model editing.ROME treats MLP layers as implicit key–value stores and locates states whose restoration recovers the correct prediction.Source: §C.2 · p. 15

C.3 Causal Tracing Protocol

  • Causal tracing compares clean and noise-corrupted executions, then restores selected clean activations to measure normalised recovery.MLP and MHA restoration uses sliding windows of 10 layers; hidden-state restoration targets individual layer–token pairs, with K = 50 used unless stated otherwise.Source: §C.3 · p. 15

D.1 PoS Grouping

  • Across nouns and verbs, localisation distributions are broadly similar, and no systematic association appears between definition length and layer distributions.The authors report only minor component-specific part-of-speech differences.Source: Figure 7 · p. 15, §D.1 · p. 16

D.2 Correlation with Definition Length

  • No systematic association between definition length and layer distributions is observed across MLP, MHA, or hidden components.The authors interpret this as evidence that localisation patterns are not confounded by input length.Source: §D.2 · p. 16

D.3 DSR-Labelled MHA and Hidden-State Traces

  • MHA recovery is concentrated at prompt tokens in middle-to-upper layers, while hidden-state recovery concentrates at the final token in upper layers.At broader aggregation levels, DSR-labelled definition tokens contribute more to MHA recovery than at the top-10 threshold.Source: §D.3 · p. 16

D.4 Alternative Definitions

  • Across alternative definitions, MLP retains a broad early-and-upper-layer pattern, while MHA and hidden-state recovery remain concentrated at the last token.Specific MLP token positions shift with lexical variation; MHA patterns are more stable across paraphrases.Source: §D.4 · p. 16

D.5 Supplementary English Results

Supplementary English traces test homonym handling and numerical concepts. The examples show that recovery patterns differ with lexical cues in definitions of “baby,” while “billion” recovery is not associated with other number tokens.

D.5 Supplementary English Results

  • The supplementary traces examine two phenomena: distinct definitions of “baby” and processing of the numerical concept “billion.”Figure 15 presents the examples as additional causal tracing results.Source: §D.5 · p. 16
  • For “baby,” definitions with direct lexical cues concentrate hidden-state recovery at the final token, whereas definitions without such cues recruit additional definition tokens.MLP recovery is more sensitive to semantically related words, while MHA maintains last-token concentration across both definitions.Source: §D.5 · p. 16
  • For “billion,” recovery is not associated with tokens containing other numerical values such as “million” or “one.”The authors describe this as treating spelled-out numbers as conceptual units rather than attending to their arithmetic content.Source: §D.5 · p. 17

D.6 Supplementary Spanish Results

Spanish traces with BERTIN GPT-J-6B reproduce the English component-localisation patterns, with somewhat broader MLP positions attributed to subword fragmentation. Aggregated English layer distributions show distinct patterns across MLP, MHA, and hidden-state restorations.

D.6 Supplementary Spanish Results

  • In Spanish, MLP recovery appears at content-bearing tokens in early layers, MHA recovery concentrates at the last token in middle-to-upper layers, and hidden-state recovery concentrates there in upper layers.The authors report no qualitative MHA differences from English and describe the overall patterns as cross-lingually stable.Source: §D.6 · p. 17, Figure 17 · p. 20
  • Spanish MLP high-impact positions are slightly broader than in English, which the authors relate to increased subword fragmentation.They give “Deliberado” splitting into “Del*,” “iber*,” and “ado*” as an example.Source: §D.6 · p. 17

D.7 Layer-Index Distributions

  • Across correctly predicted English samples, top-10 MLP states cluster in layers 0–5 and 17–21, MHA states in 12–25, and hidden states in 20–27.The distributions complement example-level traces by summarising the layer-index patterns.Source: §D.7 · p. 17, Figure 16 · p. 20
  • The figure captions organise supplementary traces by component and definiendum, while the layer-index plots compare restoration distributions across components.Figure 17 uses input tokens on the y-axis and window centre or layer index on the x-axis; Figure 16 plots layer index against normalised count.Source: Figure 17 · p. 20, Figure 16 · p. 20
Loading 2408.11827v2…