PaperlayerView exact revision on arXiv

Source-linked AI summary

Decomposing Brain-LLM Alignment with Interpretable Semantic Representations

Camila Kolling, Mariya Toneva

arXiv:2610.21508v1cs.AIcs.CL

TL;DR

LLMs predict neural responses during naturalistic language comprehension, but the linguistic information underlying that alignment is difficult to interpret. The paper uses SCRIBE and variance partitioning to characterize shared and unique brain-predictive information for LLaMA2-7B. Shared variance dominates, while both representations retain unique contributions and semantic properties have cumulative effects.

  • Problem

    Although LLMs predict neural responses during naturalistic language comprehension, brain predictivity alone does not identify which linguistic information underlies that alignment.

    Source: §1 · p. 1

  • Method

    The study uses voxelwise variance partitioning between LLaMA2-7B and SCRIBE, then tests semantic-property concreteness and cumulative property ablations.

    Source: §1 · p. 2, §3.3 · p. 3

  • Results

    Shared variance is the dominant component across the whole brain and language-related regions, while both representations retain significant unique contributions.

    Source: §4.1 · p. 4

  • Takeaways & Limitations

    For LLaMA2-7B on the Moth Radio Hour dataset, interpretable semantics characterize a substantial part of brain alignment and reveal complementary brain-predictive information beyond LLaMA2-7B.

    Source: §5 · p. 5

    The study uses one LLaMA2-7B layer, one fMRI dataset, and seven participants, so conclusions are specific to this setting; layer sensitivity and generalization to newer or larger LLMs remain untested.

    Source: §5 · p. 5

Abstract

from arXiv · show

Large language models (LLMs) predict neural responses during naturalistic language comprehension, but the linguistic information underlying this brain alignment remains difficult to interpret. We use SCRIBE, an interpretable contextual semantic representation spanning 65 neurocognitively grounded semantic properties, as a lens for decomposing the brain alignment of one LLM, LLaMA2-7B. Using variance partitioning between LLaMA2-7B and SCRIBE on naturalistic fMRI data from seven participants listening to narrative stories (Moth Radio Hour), we separate brain-predictive variance shared between the two representations from variance unique to each. We find that shared variance is the dominant component across the whole brain and language-related regions, while both representations also retain significant unique contributions. We then examine whether the concreteness of individual semantic properties relates to shared and SCRIBE-unique brain-predictive variance. Concrete properties show greater shared variance with LLaMA2-7B than abstract properties across the whole brain and most examined regions, whereas abstract properties do not show greater SCRIBE-unique variance. Finally, property ablations reveal that semantic contributions extend across multiple dimensions, with cumulative effects on both shared and SCRIBE-unique variance. Together, these results show that, for LLaMA2-7B on this dataset, a substantial portion of brain-LLM alignment is associated with interpretable semantic information, while revealing complementary brain-predictive semantic information beyond the LLaMA2-7B representation.

1 Introduction

LLMs predict neural responses during naturalistic language comprehension, but their brain alignment does not reveal which linguistic information underlies it. This study uses SCRIBE to characterize the semantic information shared with LLaMA2-7B and the information unique to each representation.

1 Introduction

  • Brain predictivity alone cannot identify which aspects of language underlie LLM–brain alignment because LLM representations jointly encode semantic, syntactic, and other information.Prior work has shown that LLMs predict neural responses during naturalistic language comprehension [11].Source: §1 · p. 1
  • SCRIBE maps words in context onto 65 interpretable semantic properties grounded in cognitive and neurobiological theories.The properties span perceptual, motor, spatial, temporal, social, cognitive, and affective dimensions.Source: §1 · p. 1
  • Variance partitioning between SCRIBE and LLaMA2-7B separates brain-predictive information shared by the representations from information unique to each.The analyses compare concreteness-related contributions and use property ablations to examine individual and cumulative semantic-property contributions.Source: §1 · p. 2

2 Related Work

Prior work has established both brain-predictive language-model representations and interpretable semantic representations. This work connects those research strands by decomposing neural variance captured by both representation types.

2 Related Work

  • Contextual language-model representations predict neural responses during language comprehension, and prior work has examined syntactic contributions to brain–LLM alignment [11] [8].Brain predictivity generally increases with model scale and performance [1].Source: §2 · p. 2
  • Feature-based semantic models use explicit, cognitively grounded dimensions to characterize semantic organization and predict neural responses.SCRIBE extends this framework to naturalistic language using LLMs as contextual semantic annotators.Source: §2 · p. 2
  • This work connects the two research strands by decomposing neural variance captured by LLM and interpretable semantic representations.Source: §2 · p. 2

3 Methodology

The methodology compares contextual LLaMA2-7B representations with SCRIBE’s interpretable semantic features using naturalistic fMRI encoding models. Variance partitioning and property analyses distinguish shared from representation-unique brain-predictive information.

3.1 Representations

  • SCRIBE represents content-bearing words with 65 contextual semantic properties, while LLaMA2-7B representations come from intermediate layer 15 and use a longer preceding-word context.SCRIBE scores average four LLM annotators and were validated against human contextual semantic judgments; LLaMA2-7B uses a 256–512-word context window, compared with SCRIBE’s 10-token window.Source: §3.1 · p. 2

3.2 Brain encoding

  • Seven participants listened to Moth Radio Hour stories during fMRI, and voxelwise ridge-regression models predicted held-out story responses from temporally aligned word representations.Analyses covered the whole brain and predefined language-related regions, restricting voxels to those with estimated noise ceiling above 0.4.Source: §3.2 · p. 3

3.3 Variance partitioning

  • Voxelwise variance partitioning fits LLaMA2-7B, SCRIBE, and concatenated encoding models, then separates squared held-out prediction correlations into shared and unique components.Component significance is assessed against block-permutation nulls; participant comparisons use paired, one-sided exact Wilcoxon signed-rank tests.Source: §3.3 · p. 3

3.3.1 Semantic property analysis

  • Concreteness analyses compare the 10 most concrete and 10 most abstract properties, while leave-one-property-out ablations rank all 65 properties by decreases in shared and SCRIBE-unique variance.Nested top-5, top-10, and top-20 property sets are then removed together; single-property decreases are treated as unique contributions given the other properties, not overall importance.Source: §3.3.1 · p. 3

4 Results

Shared variance between LLaMA2-7B and SCRIBE dominates brain-predictive variance, while both representations retain unique contributions. Concrete properties show greater shared variance than abstract properties across most regions, and cumulative ablations reveal contributions from multiple properties.

4.1 Brain–LLM alignment is largely shared with interpretable semantics

  • In the whole brain, shared variance is 0.028, or 71% of LLaMA2-7B’s explained variance and 98% of SCRIBE’s; it dominates and exceeds permutation baseline in every region.The shared component is the main overlap between the representations, while unique contributions remain.Source: §4.1 · p. 4, Figure 1 · p. 4

4.2 Concrete semantics are more strongly shared with LLaMA

  • Concrete properties yield greater shared LLaMA2-7B–SCRIBE variance than abstract properties across the whole brain and three of four language-related regions, but not significantly in LTC.Abstract properties do not explain significantly greater SCRIBE-unique variance than concrete properties in any region.Source: Figure 2 · p. 4, §4.2 · p. 5

4.3 Semantic contributions extend across multiple properties

  • Removing the top 20 properties from the shared ranking reduces shared variance by approximately 25–29% across regions; the whole-brain decrease is 27% versus 17% summed across individual leave-one-out effects.The result is consistent with correlated properties carrying redundant information that is lost when removed together.Source: Figure 2 · p. 4, §4.3 · p. 5
  • Top-set ablations also reduce SCRIBE-unique variance: whole-brain median decreases are 28–66% for unique-ranked sets and 22–60% for shared-ranked sets.All cumulative ablations significantly reduce both components in every region, with 14 of the top 20 properties overlapping between rankings.Source: Figure 2 · p. 4, §4.3 · p. 5

5 Conclusion

For LLaMA2-7B on the Moth Radio Hour dataset, much brain-predictive information is shared with interpretable semantics, while both representations retain complementary unique contributions. The conclusions are limited to one model layer, one fMRI dataset, and seven participants.

5 Conclusion

  • A substantial component of brain-predictive information is shared with interpretable semantics, while LLaMA2-7B and SCRIBE each retain complementary unique contributions.Concrete properties share more information with LLaMA2-7B than abstract properties across most regions, but concreteness does not explain SCRIBE-unique variance; ablations show cumulative contributions across multiple semantic dimensions.Source: §5 · p. 5
  • The study covers one LLaMA2-7B intermediate layer, one fMRI dataset, and seven participants, so its conclusions are specific to this setting.Sensitivity to layer choice and generalization to newer or larger models remain untested; an LLM-specific contribution cannot be fully excluded because two SCRIBE annotators share LLaMA2-7B’s model family.Source: §5 · p. 5

A Concrete and abstract properties

Table 1 identifies the 10 most concrete and 10 most abstract SCRIBE properties according to concreteness norms; higher scores indicate greater concreteness.

A Concrete and abstract properties

  • Table 1 lists the 10 most concrete and 10 most abstract SCRIBE properties, ranked using norms in which higher scores indicate greater concreteness.The norms are from Brysbaert et al. [3].Source: Table 1 · p. 7

B Correlations between SCRIBE properties

SCRIBE properties are correlated and form related-dimension clusters, which makes individual leave-one-out decreases small. Correlations largely reflect semantic structure, although word rate raises TR-level correlations overall.

B Correlations between SCRIBE properties

  • Mean absolute correlation between the 65 SCRIBE properties is 0.23, and the median variance inflation factor is 5.3; correlated dimensions make individual leave-one-out decreases small.Examples of related clusters include visual form and social communication; removing a property can leave some of its information available through correlated properties.Source: B Correlations between SCRIBE properties · p. 7
  • Word- and TR-level property-correlation matrices are highly similar (r = 0.81), rising to r = 0.95 after removing word rate, with the same strongest pairs.Word rate raises overall mean correlation from 0.08 across words to 0.22 across TRs; Speech–Human is 0.84 at the TR level versus 0.68 across words.Source: B Correlations between SCRIBE properties · p. 7
  • Table 2 compares SCRIBE-property correlations across content words, across TRs, and across TRs after regressing out word rate.The table’s caption defines the three comparison levels but provides no specific cell values.Source: Table 2 · p. 7
  • Figure 3 orders Pearson correlations between SCRIBE properties at the TR level using hierarchical clustering.Properties are ordered by average linkage on 1 − r.Source: Figure 3 · p. 8
  • Figure 4 compares property correlations across content words, across TRs, and across TRs after regressing out word rate, using the same property order.The pattern is largely semantic, while word rate adds a positive shift to TR-level correlations overall.Source: B Correlations between SCRIBE properties · p. 7, Figure 4 · p. 8

C Leave-one-property-out rankings

Leave-one-property-out rankings identify properties whose removal most decreases shared or SCRIBE-unique variance, but the rankings reflect unique contributions given the other correlated properties.

C Leave-one-property-out rankings

  • Number, Taste, and Surprised rank highest for shared variance, while Number, Landmark, and Scene rank highest for SCRIBE-unique variance.The listed rankings extend to 20 properties for each component.Source: C Leave-one-property-out rankings · p. 9
  • Figure 5 ranks the top 20 properties by mean whole-brain decrease in shared variance and SCRIBE-unique variance when each property is removed individually.Error bars show SEM across participants, and numbers indicate participants showing a decrease.Source: Figure 5 · p. 9

Related papers

Loading 2610.21508v1…