Source-linked AI summary
Linking artificial and human neural representations of language
Jon Gauthier, Roger Levy
TL;DR
The paper asks what sentence information is shared between human brain activity and language-model representations. It compares brain decoding across task-optimized models and finds that syntax-light scrambled-language-model representations improve decoding, while retaining some syntactic information that is decodable from the models.
Problem
The paper asks which syntactic features are responsible for the improved brain-decoding performance of LM-scrambled models.
Source: §4 · p. 9Method
The authors compare sentence representations from different models by learning and evaluating linear mappings from fMRI activity to model representations, then probe model syntax.
Source: §2.3 · p. 5, §3.2.2 · p. 7Results
Scrambled language modeling yields improved brain decoding and coherent pairwise sentence-similarity predictions, alongside a clear loss of syntactic information; some local grammatical dependencies remain decodable.
Source: §4 · p. 8Takeaways & Limitations
The models are candidate algorithmic descriptions of sentence representation that can be probed to investigate why brain decoding succeeds or fails.
Source: §4 · p. 9The linear linking hypothesis assumes syntactic structure is encoded entirely in the linear geometry of brains and neural networks, although some syntactic information may not be readable by a linear decoder.
Source: §4 · p. 9
Abstract
from arXiv · showhide
What information from an act of sentence understanding is robustly represented in the human brain? We investigate this question by comparing sentence encoding models on a brain decoding task, where the sentence that an experimental participant has seen must be predicted from the fMRI signal evoked by the sentence. We take a pre-trained BERT architecture as a baseline sentence encoding model and fine-tune it on a variety of natural language understanding (NLU) tasks, asking which lead to improvements in brain-decoding performance. We find that none of the sentence encoding tasks tested yield significant increases in brain decoding performance. Through further task ablations and representational analyses, we find that tasks which produce syntax-light representations yield significant improvements in brain decoding performance. Our results constrain the space of NLU models that could best account for human neural representations of language, but also suggest limits on the possibility of decoding fine-grained syntactic information from fMRI human neuroimaging.
1 Related work
Earlier work linked neural activity to increasingly rich linguistic representations, including sentence representations and models incorporating longer-term context. This paper broadens that paradigm across task-optimized models and probes which representational contents relate to brain-decoding performance.
1 Related work
- Prior studies showed distributional word representations predict brain activations for isolated words, extended this work to sentence stimuli, and found neural-network language models better accounted for brain activation using more complex word and sentence meaning representations.Source: §1 · p. 2
- Figure 1 depicts the approach of using sentence-evoked human brain activations to predict neural-network representations of those same sentences.Source: Figure 1 · p. 2
- This paper extends the brain-decoding paradigm by studying a broad class of task-optimal models and probing which representational contents account for their performance differences.Earlier studies found that optimizing models for different objectives could yield substantial differences in brain decoding performance.Source: §1 · p. 2
2 Methods
The study maps sentence-evoked fMRI activity to representations from language models, using a common BERT architecture fine-tuned on varied tasks. It compares those task-dependent representations with context-insensitive GloVe vectors under a linear decoding evaluation.
2 Methods
- Linear regression maps each subject’s brain activations to model representations of the same 384 sentences, testing whether model representations are linearly recoverable from brain activity.The framework uses decoding performance to assess shared structure between brain and model representations, while recognizing that fMRI and linear decoding may limit what is recoverable.Source: §2 · pp. 2–3
2.1 Human neuroimaging data
- Eight adults read 384 visually presented natural-language sentences about natural kinds while fMRI recorded sentence-evoked activity, compressed to 256 dimensions per subject with PCA.Each original response was approximately 200,000-dimensional; the PCA projection retained more than 95% of within-subject variance.Source: §2.1 · p. 3, §2.2 · p. 3
2.2 Sentence representation models
- Sentence representations come from a shared BERT architecture optimized for different tasks, allowing comparison of task-driven representational content rather than neural-network architectures.BERT’s final-layer [CLS] representation supplies a single vector for each sentence.Source: §2.2 · pp. 3–4
2.2.1 Fine-tuned models
- BERT is fine-tuned on shared NLU classification tasks and custom cloze variants, including word-shuffled language modeling and masked-word part-of-speech prediction.The custom tasks are designed to select against fine-grained syntax or semantics; a continued-training language-modeling task serves as a control.Source: §2.2.1 · pp. 4–5
2.2.2 Word vector baseline
- The GloVe baseline averages context-insensitive word vectors, providing a sentence representation that does not depend on sentential context.Prior studies had also used these vectors as sentence-meaning representations.Source: §2.2.2 · p. 5
2.3 Brain decoding
- For each subject and target representation, a linear map is fit with regularized regression and nested 8-fold cross-validation, then evaluated using MSE and average rank.MSE tests exact matching of representational geometry; average rank tests whether brain activity supports meaning contrasts among the 384 sentences.Source: §2.3 · p. 5
3 Results
Standard NLU fine-tuning worsened brain-decoding error, while paragraph-scrambled language modeling improved it. Representational analyses linked decoding differences with syntax-light representations and found that decoding performance changes rapidly before stabilizing.
3 Results
- LM and LM-pos results were ambiguous: both reduced MSE but showed no significant AR change relative to BERT.Source: §3 · p. 6
3.2 Representational analysis
- Differences in syntactic-representation fidelity can explain some major qualitative differences in brain-decoding performance across models.Source: §3.2 · p. 6
3.2.1 Representational similarity
- Language-modeling runs, especially the scrambled tasks, were coherent in their sentence-distance predictions; other models were less coherent across runs.The scrambled LM tasks also produced reliable brain-decoding improvements.Source: §3.2.1 · p. 6
3 Results
- Standard NLU tasks increased error on both MSE and AR versus pretrained BERT, while LM-scrambled-para decreased both versus BERT and GloVe.Figure 3 compares fine-tuned BERT models and GloVe with pretrained BERT performance.Source: §3 · p. 6, Figure 3 · p. 7
3.1 Learning dynamics
- Brain-decoding performance rapidly diverged during fine-tuning and remained mostly stable after about 100 steps.The rapid performance changes generally coincided with rapid changes in task performance.Source: §3.1 · p. 6, Figure 4 · p. 7
3.2.2 Syntactic probe
- Structural probes learn a transformation from contextual word-representation distances to dependency-tree distances, then reconstruct parses scored by unlabeled attachment.Source: §3.2.2 · pp. 7–8
- LM-scrambled and LM-scrambled-para probe performance worsened during fine-tuning but stayed above GloVe; a representative LM-scrambled parse made better local attachment guesses than GloVe.The local-attachment comparison was observed despite LM-scrambled training not using relative word positions.Source: §3.2.2 · p. 8
4 Discussion
Scrambled language-modeling tasks best matched brain-activation structure among the tested models, while retaining some syntactic information despite a clear loss of it. The discussion cautions that linear decoding and the study’s fMRI data limit what can be inferred about syntactic representations.
Discussion
- Scrambled language-modeling tasks best matched brain-activation structure among tested models, improving brain decoding while yielding coherent sentence-similarity predictions.These changes coincided with a clear loss of syntactic information, although minimal local grammatical-dependency information remained decodable.Source: §4 · p. 8
- The results do not indicate that human neural representations lack syntactic information; the authors note that fMRI may be too temporally coarse to detect syntactic computations.They frame their findings as hypotheses about syntactic information preserved at coarse temporal resolutions, including in relation to prior findings.Source: §4 · pp. 8–9
- The LM-scrambled models still retained some syntactic information or correlates, outperforming a baseline in predicting sentence syntactic parses.The authors consider these results compatible with specific syntactic features remaining represented in the brain at fMRI timescales.Source: §4 · p. 9
- Unlike prior brain-mapping studies that detect summary features but do not provide complete proposals of syntactic processing, the models here are candidate algorithms that can be probed to explain decoding outcomes.The authors propose further probing and ablated datasets to narrow down the specific syntactic features behind LM-scrambled performance.Source: §4 · p. 9
Limitations of the linking hypothesis
- The linear linking hypothesis assumes syntactic structure is encoded entirely in the linear geometry of neural-network and brain representations.Some syntactic information may remain in fMRI but be unreadable by a linear decoder; the authors call for more complex linking transforms.Source: §4 · p. 9
Limitations of the data
- The fMRI data came from subjects asked to read sentences and think about their meaning, a vague task that may have encouraged superficial engagement.The authors suggest such shallow representations might favor simple tasks such as LM-scrambled and LM-scrambled-para, and recommend comparing different behavioral tasks.Source: §4 · p. 9
A Supplemental figures
The supplemental figures show brain-decoding performance changes after fine-tuning, grouped in two different ways and evaluated with two metrics.
Supplemental figures
- Figure 8 shows changes in brain-decoding performance after fine-tuning, grouped by subject and evaluated using both mean squared error and rank metrics.Source: A Supplemental figures · p. 12
B Hyperparameters
The appendix lists hyperparameter tables for fine-tuning and syntactic probing; the supplied passages identify their scope and sources, not parameter values.
Hyperparameters
- Table 3 lists fine-tuning hyperparameters shared across runs for all tasks, mostly following Devlin et al. (2018).The passage does not provide the hyperparameter values.Source: Table 3 · p. 12
- Table 4 lists syntactic-probe hyperparameters, following the defaults of Hewitt and Manning (2019).The passage does not provide the hyperparameter values.Source: Table 4 · p. 12
C Custom task information
The appendix describes the custom language-modeling task data and part-of-speech setup, and points to examples and learning curves. Supplemental figures describe decoding changes by subject and by model.
Custom task information
- Each language-modeling dataset used 1,000,000 training sentences and 100,000 development and test sentences sampled from concatenated Toronto Books Corpus documents.Oversized datasets were intended to make same-model runs unlikely to see similar training samples.Source: C Custom task information · p. 12
- The part-of-speech task used spaCy tags, with 49 unique tags forming a 49-way classification task and the typical cloze masking procedure.Across tasks, the secondary BERT objective predicted whether two sentences were adjacent in a source document; for scrambling, input sentences were independently shuffled.Source: C Custom task information · p. 12
- Table 5 provides custom-task training examples, while Figure 10 shows training-set loss and evaluation-set accuracy learning curves.The passages identify these materials but do not report curve values or task comparisons.Source: C Custom task information · p. 12, Figure 10 · p. 14, Table 5 · p. 15
- Figure 8 shows within-subject changes in brain-decoding performance by model relative to pre-trained BERT, with 95% confidence intervals pooled across up to eight decoder runs.The figure covers mean squared error and rank metrics.Source: Figure 8 · p. 13
- Figure 9 shows within-model changes by subject relative to that subject’s pre-trained BERT performance, with 95% confidence intervals pooled across up to eight decoder runs.The figure covers mean squared error and rank metrics.Source: Figure 9 · p. 14
- Table 5’s examples include ordinary cloze sentences and scrambled inputs with masked tokens, illustrating the kinds of training examples shown.The supplied passages do not specify the full task definitions for all examples.Source: Table 5 · p. 15, C Custom task information · p. 15