PaperlayerView exact revision on arXiv

Source-linked AI summary

Joint processing of linguistic properties in brains and language models

Subba Reddy Oota, Manish Gupta, Mariya Toneva

arXiv:2212.08094v2cs.CLq-bio.NC

TL;DR

It remains unclear which linguistic information underlies the observed alignment between language models and brain recordings, despite prior evidence of predictive alignment. This paper removes linguistic properties from model representations and tests the resulting fMRI alignment; each removal reduces alignment, with syntactic properties most affecting its layerwise trend.

  • Problem

    Although language models predict brain recordings during language comprehension, it remains unclear which linguistic information underlies that alignment.

    Source: §1 · p. 1

  • Method

    The authors remove surface, syntactic, and semantic properties from BERT representations and use the resulting features in voxelwise encoding models to predict fMRI responses.

    Source: §3 · p. 4, §4 · p. 5, §6 · p. 10

  • Results

    Removing each tested property significantly decreases alignment across BERT layers; Top Constituents has the largest effect on the trend across language regions, while Tree Depth strongly affects temporal and frontal regions.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    The findings provide evidence that specific linguistic properties contribute to brain–language-model alignment, while substantial alignment remains after property removal.

    Source: §6 · p. 10

    Removing a property also removes information correlated with it in the dataset, and substantial alignment remains after all tested properties are removed.

    Source: §6 · p. 10

Abstract

from arXiv · show

Language models have been shown to be very effective in predicting brain recordings of subjects experiencing complex language stimuli. For a deeper understanding of this alignment, it is important to understand the correspondence between the detailed processing of linguistic information by the human brain versus language models. We investigate this correspondence via a direct approach, in which we eliminate information related to specific linguistic properties in the language model representations and observe how this intervention affects the alignment with fMRI brain recordings obtained while participants listened to a story. We investigate a range of linguistic properties (surface, syntactic, and semantic) and find that the elimination of each one results in a significant decrease in brain alignment. Specifically, we find that syntactic properties (i.e. Top Constituents and Tree Depth) have the largest effect on the trend of brain alignment across model layers. These findings provide clear evidence for the role of specific linguistic information in the alignment between brain and language models, and open new avenues for mapping the joint information processing in both systems. We make the code publicly available [https://github.com/subbareddy248/linguistic-properties-brain-alignment].

1 Introduction

The paper directly tests which linguistic properties contribute to brain–language-model alignment, addressing a gap left by coarse alignment results and separate studies of linguistic processing. Removing each tested property significantly reduces alignment, with syntactic properties most affecting its layerwise trend.

1 Introduction

  • The study asks which linguistic information underlies brain–language-model alignment, since prior work established coarse similarities but not how the systems align in information processing.Prior research separately used probing tasks to study properties encoded in language models and controlled stimuli to study brain processing.Source: §1 · pp. 1–2
  • The authors remove information about specific linguistic properties from BERT representations and compare brain alignment before and after removal using fMRI recordings from 18 people listening to a story.They examine a range of properties previously shown to be represented in pretrained BERT.Source: §1 · p. 2, Figure 1 · p. 2
  • Removing each tested linguistic property significantly decreases brain alignment across BERT layers; Top Constituents most affects the layerwise trend across language regions, while Tree Depth has strong effects in temporal and frontal regions.The authors report similar results with GPT2.Source: §1 · p. 2

2 Related Work

This work extends brain-alignment research by testing the contribution of specific linguistic properties directly, complementing studies of word meaning, probing, and attention-head performance.

2 Related Work

  • Building on a residual approach that removed individual words to study supra-word meaning, this work examines linguistic properties across model layers.The authors describe their approach as complementary to work correlating BERT attention-head syntactic-task performance with brain encoding performance.Source: §2 · p. 3
  • Unlike NLP studies that map linguistic properties across language-model layers, this work tests how those properties affect prediction of brain recordings.The cited NLP work investigates which properties are encoded in model representations.Source: §2 · p. 3
  • The approach complements prior studies linking brain responses to word embeddings, sentence models, recurrent networks, and Transformer representations.The authors frame their approach as a way to further understand the reasons for observed brain alignment.Source: §2 · p. 3

3 Dataset Curation

The dataset curation assembles story-based fMRI, BERT word features, and annotations for six probing tasks, with preprocessing to match linguistic features to fMRI samples.

3 Dataset Curation

  • 18 participants’ fMRI data from the “21st year” story provide the brain recordings, with 2226 samples per participant.The data were preprocessed and projected to surface space, and brain maps use the Glasser Atlas.Source: §3 · p. 3
  • BERT word representations come from each hidden layer of the 12-layer, 768-dimensional pretrained base model, using a context of up to 20 previous words.GPT-2 features are extracted identically for the comparison analyses.Source: §3 · pp. 3–4
  • Because fMRI acquisition is slower than word presentation, stimulus features are downsampled with a 3-lobed Lanczos filter to produce one chunk embedding per TR.The TR is 1.5 seconds.Source: §3 · p. 4
  • The six probing tasks cover sentence length, tree depth, top constituents, tense, subject number, and object number across surface, syntactic, and semantic categories.Other tasks were excluded because their labels were constant for this brain dataset.Source: §3 · p. 4
  • Labels for Sentence Length, TreeDepth, and TopConstituents were regrouped to reduce skew from frequent class co-occurrences across properties.The regrouping addresses the risk that removing one property could affect performance for another property.Source: §3 · p. 5
  • Sentence-level labels use Stanford CoreNLP stanza annotations, while word-level annotations assign each sentence’s label to its words except for sentence length.Source: §3 · p. 5
  • Subject Number and Object Number labels are highly correlated; the other task pairs show only small positive or negative correlations.The task similarity matrix uses Pearson correlations between task labels across the six tasks.Source: Figure 2 · p. 4, §3 · p. 5

4 Methodology

The methodology removes linguistic-property information from BERT features using ridge-regression residuals, then tests how the resulting representations predict fMRI responses. It evaluates voxelwise prediction with banded ridge regression and Pearson correlation.

4 Methodology

  • Ridge regression predicts word features from probing-task labels; subtracting those predictions yields residuals that linearly remove the property’s contribution to brain prediction.The authors also report similar results with Iterative Null Space Projection (INLP), but use ridge regression in the main paper for simplicity.Source: §4 · p. 5
  • The authors also report similar property-removal results for GPT-2 representations.The reported GPT-2 results are in Appendix Table 11.Source: §4 · p. 5
  • Banded ridge regression uses layerwise BERT features and property-removed residuals to predict fMRI responses for individual voxels.Features and fMRI responses are z-scored separately, and cross-validation selects the regularization parameter for each feature space.Source: §4 · p. 5
  • Pearson correlation between actual and predicted voxel response vectors is the brain-alignment evaluation metric.The paper denotes the vectors as Y and Ŷ and computes PC as corr(Y, Ŷ).Source: §4 · p. 6
  • The ridge-regression reproducibility setup uses MSE loss, four-fold cross-validation, and L2 decay tuned over 10^-1 to 10^-3.The best λ is selected on validation data.Source: §4 · p. 6

5 Results

Removing linguistic properties from BERT reduces brain alignment, while layer-wise analyses show that syntactic and semantic properties contribute differently across the whole brain and specific regions. The results also indicate that the removal approach affects decoding of its target property while leaving most other properties largely unaffected.

5.1 Successful removal of linguistic properties from pretrained BERT

  • Probing confirms that BERT represents surface information in initial-to-middle layers, syntax in middle layers, and semantics in middle-to-top layers; removing each property reduces its task performance.Logistic-regression probes test whether task labels remain decodable from the residual representations.Source: §5.1 · p. 6
  • Removing one property generally does not substantially affect decoding performance for other properties.The reported exceptions are not specified in the supplied passage.Source: §5.1 · p. 6

5.2 Removal of linguistic properties significantly decreases brain alignment across all layers

  • Removing each linguistic property significantly reduces BERT’s brain alignment across layers, while a random-vector removal does not significantly affect alignment.Residual alignment after property removal exceeds alignment with the random vector, suggesting that other properties also contribute.Source: §5.2 · p. 7, Figure 3 · p. 7

5.3 Decoding task performance vs brain alignment

  • Top Constituents and Tree Depth have the strongest effects on the whole-brain alignment trend across BERT layers, while surface and semantic properties have moderate effects.The correlations compare changes in decoding performance with corresponding changes in brain alignment.Source: §5.3 · p. 8, Table 2 · p. 8
  • Top Constituents affects alignment across language regions, especially IFG, while Tree Depth has strong effects in temporal and frontal regions; semantic properties affect PCC, and Sentence Length affects IFG and MFG.Sentence Length’s effects in IFG and MFG are smaller than those of syntactic properties.Source: §5.3 · pp. 8–9
  • Across subregions, semantic properties shape trends in many ATL, PTL, and AG areas, while syntactic properties have the highest effect in IFG; Object Number matters across many ATL and PTL subregions.Top Constituents is strong in only three IFG subregions, and Tree Depth affects IFSp and IFJa more strongly than Top Constituents.Source: Table 3 · p. 8, §5.3 · p. 9
  • Voxel-wise patterns show Tree Depth, Top Constituents, and Object Number more localized to canonical left-hemisphere language regions and more distributed on the right; Object Number has a larger left-hemisphere effect.Removing all properties produces different region-level maps from removing individual properties, with little remaining correlation in key language regions.Source: §5.3 · pp. 9–10, Figure 4 · p. 9

6 Discussion and Conclusion

The paper proposes removing linguistic information from language-model representations to evaluate joint processing with brain recordings, finding that linguistic properties affect alignment and that syntactic properties particularly shape its layer-wise trend. It also identifies limits of the approach and describes possible applications in AI engineering, neuroscience, and interpretability.

6 Discussion and Conclusion

  • Removing linguistic properties from BERT and GPT-2 significantly decreases brain alignment across model layers, and Tree Depth and Top Constituents most shape BERT’s whole-brain alignment trend.Top Constituents has the largest effect across language regions, while Tree Depth strongly affects a smaller set of temporal and frontal regions.Source: §6 · p. 10
  • Removal also deletes dataset-correlated information, leaves substantial alignment unexplained, and was tested using English stimuli and English-trained models.The authors expect possible generalization to syntactically close languages but state that this needs exploration.Source: §6 · p. 10
  • The approach may guide linguistic feature selection, transfer learning, and cognitively plausible AI architectures, while giving neuroscientists more control over language models as model organisms.The authors also hope it will contribute to brain-signal-based model interpretability.Source: §6 · p. 10

A Distribution of Class Labels Across Each Probing Task

The section reports how probing-task labels are distributed and describes balancing classes for three tasks. It also uses pairwise common-sample counts to assess balance across tasks.

A Distribution of Class Labels Across Each Probing Task

B INLP Projection vs. Our Method

The section compares the removal method with INLP and tests whether removing one property affects probing performance for another. It reports similar results to INLP and limited cross-task effects, with TreeDepth and TopConstituents as exceptions.

B INLP Projection vs. Our Method

B.1 Probing Analysis While Removing Information about a Property and Testing Another

  • Removing one property generally did not affect probing performance for other properties, except for TreeDepth and TopConstituents.Source: §B.1 · p. 17
  • Tables 5 and 6 report layer-wise probing performance for Sentence Length and TreeDepth, respectively, before and after removal.Source: Table 5 · p. 17, Table 6 · p. 17

C Layer-wise Whole Brain Analysis before/after removal of linguistic properties.

The whole-brain analysis compares pretrained BERT alignment before and after removing each tested linguistic property. Alignment is significantly worse mainly in the middle-to-late layers after removal.

C Layer-wise Whole Brain Analysis before/after removal of linguistic properties.

D Layer-wise Language ROIs Analysis before/after removal of linguistic properties.

The analysis examines regions of interest associated with language comprehension and word semantics. Across language regions, removing linguistic properties significantly decreases alignment across all layers.

D Layer-wise Language ROIs Analysis before/after removal of linguistic properties.

E Layer-wise Language sub-ROIs Analysis before/after removal of linguistic properties.

Removing linguistic properties from pretrained BERT worsens alignment in the examined language sub-regions. The analysis focuses on local trends because broader language regions may conceal functional variation across their voxels.

E Layer-wise Language sub-ROIs Analysis before/after removal of linguistic properties.

  • Language sub-regions align significantly worse after linguistic properties are removed from pretrained BERT.The examined sub-regions include 44, 45, IFJa, IFSp, STGa, STSdp, STSda, A5, PSL, and STV.Source: E Layer-wise Language sub-ROIs… · p. 19
  • Because language regions can vary in function across voxels, aggregate analyses may mask nuanced effects.The analysis examines sub-regions to capture this variety and reports their correlations separately.Source: E Layer-wise Language sub-ROIs… · p. 19
  • Figure 7 compares layer-wise BERT alignment before and after removing probing-task information, with red asterisks marking significant reductions.Figure 8 shows average Pearson correlations across eight language regions for selected properties.Source: Figure 7 · p. 19, Figure 8 · p. 20

F Probing Results: GPT2

In GPT2, removing a linguistic property significantly reduces brain alignment across layers. Probing results show a hierarchy of linguistic signals across model layers, similar to BERT.

F Probing Results: GPT2

  • GPT2 brain alignment is significantly reduced across all layers after removing a linguistic property.Figure 11 compares layer-wise performance before and after removing TopConstituents.Source: F Probing Results: GPT2 · p. 20, Figure 11 · p. 22
  • GPT2 probing results show surface information in initial-to-middle layers, syntax in middle layers, and semantics in middle-to-top layers, similar to BERT.Removal of each linguistic property reduces probing-task performance across all layers.Source: F Probing Results: GPT2 · p. 20
  • Table 11 reports word-level probing-task performance for each GPT2 layer before and after removing each linguistic property.Source: Table 11 · p. 21
Loading 2212.08094v2…