PaperlayerView exact revision on arXiv

Source-linked AI summary

The Score Is Not the Structure: Brain Alignment and Cross-Lingual Transfer

Saman Rahbar

arXiv:2610.03827v4cs.CLcs.AIcs.LGq-bio.NC

TL;DR

The paper asks whether similarity scores for brain alignment and cross-lingual structure measure the correspondence they are taken to represent. It audits those scores by removing correspondence and checking measurement and inference, finding that much of the score survives controls and no tested utility benefit is detected.

  • Problem

    Similarity scores are often treated as evidence that models share meaningful, usable structure with the brain or across languages, but the score may not measure the intended correspondence.

    Source: §1 · p. 1

  • Method

    The paper audits correspondence by removing the match, checking instrument reliability across comparisons, and counting independent units before testing downstream utility.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    Across both settings, much of the correspondence score survives controls, and neither yields a detectable downstream or distance-graded benefit from the surviving correspondence.

    Source: §6 · p. 8, §7 · p. 9

  • Takeaways & Limitations

    A score's magnitude is not supported as a direct measure of correspondence unless it is checked against controls that remove the correspondence it is meant to capture.

    Source: §1 · p. 1, §6 · p. 8

    In the cross-lingual sample, probe reliability and typological distance cannot be disentangled because exclusions also truncate the distance range.

    Source: §4 · p. 5

Abstract

from arXiv · show

Similarity scores are often offered as evidence that a model shares structure with the brain or across languages. We ask what such a score reads when that structure is removed, or when the instrument measuring it does not work, in two settings. Across seventeen languages, a grammaticality probe transfers worse between more distant languages (r = -0.66). But the probe's own accuracy falls along the same axis and is at chance in four languages, four of the five most distant. Dropping them halves the explained variance, and because it also narrows the range of distances, the design cannot say how much of the gradient is the instrument. Counting the 272 language pairs as independent gives p = 0.0006 for a steering effect that is null when the seventeen languages are the unit (p = 0.155). In brain alignment, a training objective raises a language model's similarity to fMRI responses from 0.10 to 0.34 (reliability ceiling 0.54), yet targets with the brain correspondence destroyed still reach 0.31. A shuffled target scores about k/n against the real one, for rank k and n sentences, a baseline that can be computed before any model is trained. Both interventions work, yet we detect no distance-graded steering effect and no syntactic benefit from alignment. Before reading a correspondence score, measure how much of it survives without the correspondence.

1 Introduction

The paper tests whether correspondence scores measure the structure they are taken to represent, using checks for score insensitivity, unreliable instruments, and miscounted units. Across cross-lingual transfer and brain alignment, the audits expose limits on interpreting score magnitudes and find no detectable benefit from the tested interventions.

1 Introduction

  • The paper proposes three inexpensive checks before testing usefulness: destroy correspondence and re-score, verify instrument reliability across comparisons, and count independent units correctly.It applies the checks in two settings and reports no detectable benefit from the surviving correspondence in either.Source: §1 · p. 2
  • Probe accuracy falls with typological distance (r = −0.74) and is at chance in four of seventeen languages, confounding the transfer gradient with measurement quality.The four languages are among the most distant, so the gradient cannot be separated from the probe's reliability along that axis.Source: §1 · p. 2
  • Counting 272 language pairs rather than languages would turn a null steering result into p = 0.0006, illustrating the risk of treating dependent observations as independent.The paper's contribution summary contrasts this with the language-level null.Source: §1 · p. 2
  • The audit finds that much of the brain-alignment score survives destroyed correspondence, while cross-lingual transfer estimates include chance-level probes and pairwise counting can turn a null into apparent significance.The paper also reports no detectable downstream or distance-graded benefit from the interventions.Source: §1 · p. 2
  • A permuted brain target recovers 83–92% of the alignment score, and the target-to-target baseline follows k/n, computable before any model is trained.The contribution summary presents these as evidence that score magnitude can persist without the intended brain correspondence.Source: §1 · p. 2

2 Related work

Prior work studies brain alignment, cognitively motivated model structure, cross-lingual transfer, and activation steering, while the authors identify measurement comparability and correspondence-destroying controls as less examined parts of the evidence. They distinguish cross-language probe reliability from within-language selectivity and note that they have not surveyed the literature systematically.

2 Related work

  • Prior work reports brain alignment mainly as post-hoc evaluation [Zhang et al., 2026], while brain-inspired redesigns and inductive-bias studies argue for cognitively motivated structure without matched tests of brain-derived training signals.The passage cites brain-inspired redesigns [Zhao et al., 2025, Pezzulo et al., 2026] and inductive-bias studies [Papadimitriou and Jurafsky, 2023, Feinman and Lake, 2018, Dijujin et al., 2025].Source: §2 · p. 2
  • Cross-lingual work finds shared agreement structure [Mueller et al., 2020] and distance-linked transfer [Littell et al., 2017, Lin et al., 2019], while lexical normalization reports no reliable distance relation for the best co-trained language count [Rahbar, 2026a].Activation steering manipulates behavior along learned directions.Source: §2 · p. 2
  • Probe selectivity tests whether a probe reads representations rather than memorizing labels, but cross-language regression additionally assumes comparable absolute reliability across languages.The authors say they did not find this instrument check in the work they build on, while cautioning that their literature survey was not systematic.Source: §2 · p. 2
  • The audit complements work on CKA sensitivity [Kornblith et al., 2019] by measuring score survival after correspondence is destroyed, and uses entity permutation for non-independent pairwise comparisons [Mantel, 1967].A related brain-imaging negative control finds scanner site decodable before pretraining [Rahbar, 2026c].Source: §2 · p. 2

3 Auditing a correspondence measure

The audit tests whether correspondence scores reflect content rather than incidental properties, whether their instruments are comparably reliable, and whether inference counts independent units. Only after these checks does it test whether exploiting the surviving correspondence changes downstream behavior.

3 Auditing a correspondence measure

  • The audit first destroys correspondence while preserving incidental properties, then checks instrument reliability, and finally permutes independent entities rather than pairwise observations.Content controls include shuffled targets, dimension-matched noise, and random directions; these test how much score remains without the intended match.Source: §3 · p. 3
  • In the cross-lingual study, random-direction steering is the content ablation, within-language probe accuracy is the instrument check, and language-level permutation is the inference check.In brain alignment, shuffled and random targets serve as ablations, the subject-split noise ceiling checks reliability, and pairing over shared seeds checks inference.Source: §3 · p. 3
  • The decisive audit differs by setting: instrument reliability dominates the cross-lingual study, whereas content ablation dominates brain alignment.The method therefore treats the same three-part audit as setting-specific in which check proves decisive.Source: §3 · p. 3
  • Downstream behavior is tested only after the audit, so a null is interpretable only if the intervention installed the content and the score measured it.The outcomes of the checks are summarized for both studies in Table 1.Source: §3 · p. 3

4 Study 1: Cross-lingual transfer and steering

Probe transfer declines with typological distance, but probe reliability declines along the same axis, complicating that gradient. Steering works within and across languages, yet the analysis finds no supported distance-graded steering effect.

4 Study 1: Cross-lingual transfer and steering

  • r = −0.66 (R2=0.43): probe transfer accuracy decreases with typological distance across the seventeen-language sample.Permuting language labels preserves the transfer association (p=10−4).Source: §4 · pp. 4–5
  • r = −0.74: probe accuracy also falls with mean typological distance, and four of seventeen probes are at or below chance.Among the thirteen languages whose probes clear chance, transfer has r = −0.47 (R2=0.22), halving explained variance.Source: §4 · p. 4, Figure 1 · p. 4
  • p = 0.0006 for the steering slope when 272 pairs are treated as independent, versus p = 0.155 when seventeen languages are the units.Each language appears in 32 pairs, so pairwise permutation treats dependent observations as independent.Source: Figure 2 · p. 5, §4 · p. 5
  • The four failed probes also rank among the most distant languages, so their removal narrows the distance range and the sample cannot separate range restriction from probe validity.The authors therefore do not claim that half the transfer gradient is measurement artifact.Source: §4 · p. 5
  • +6.21 is the average within-language steering shift; cross-language steering beats the random baseline by +2.85 nats on 223 of 272 pairs.The within-language shift is positive in sixteen of seventeen languages, and the cross-language advantage is positive for all seventeen source languages when averaged within source.Source: §4 · p. 5
  • The source-specific steering advantage does not survive language-level testing: its distance relationship is r = −0.22 (p = 0.155).The corrected relationship is not significant when the seventeen languages, rather than the 272 pairs, are the units.Source: Figure 2 · p. 5, §4 · p. 6

5 Study 2: Brain-geometry alignment and syntactic generalization

Brain alignment strongly raises similarity to the fMRI target, but most of that score survives when brain correspondence is destroyed. The alignment produces no detectable BLiMP benefit, within the tested setup.

5 Study 2: Brain-geometry alignment and syntactic generalization

  • 0.337 versus 0.095: alignment raises end-of-training CKA to the fMRI target, against a subject-split reliability ceiling of approximately 0.54.The CKA comparison uses the same 627 sentences used for fine-tuning, so it does not measure generalization to unseen stimuli.Source: §5 · p. 6
  • 0.31: shuffled and rank-matched Gaussian targets each recover 83–92% of the brain-aligned score, while brain-specific increments are +0.028, +0.068, and +0.064 across model sizes.The increments are 5–13% of the reliability ceiling; they are reliably nonzero, indicating that brain-specific structure was installed despite its small share of the score.Source: §5 · pp. 6–7, Figure 3 · p. 8
  • 0.204 and 0.186: shuffled and noise targets score against the real target, matching k/n across tested ranks while carrying no correspondence.The pattern was measured at four ranks on one target, not proven as an identity; the authors note it may also depend on the target spectrum.Source: §5 · p. 7
  • 74.5% versus 74.1%: at 160M, brain alignment does not significantly improve BLiMP accuracy over LM-only, shuffled, or random controls; the pattern holds at larger scales.The 160M difference is 0.48 points (p=0.36), and equivalence testing bounds the true effect within [−0.4, +1.4] BLiMP points.Source: §5 · p. 7, Table 2 · p. 8
  • The null bounds utility for one 627-sentence fMRI dataset, a rank-128 mean-pooled target, and one similarity objective; model-side recovery across ranks remains untested.The authors distinguish this bound on the objective from the looser bound on brain content, since only 5–13% of the ceiling was brain-specific.Source: §5 · p. 7

6 Discussion

The two audits show how a correspondence score can be inflated when its contaminant covaries with the target phenomenon, making the score appear informative. The cross-lingual analysis also cannot isolate how much of its gradient reflects probe reliability, and the paper identifies limits of both studies’ operationalizations.

6 Discussion

  • 64 of 272 transfer estimates rely on a chance-level probe, while a permuted fMRI target reproduces most of the brain-alignment score.The underlying phenomena are present, but the authors argue that the raw score magnitudes are not proportional to correspondence.Source: §6 · p. 8
  • A contaminant that covaries with the signal can inflate an apparent effect rather than look like noise, so a tight confidence interval may still measure the wrong quantity.The discussion links second-order structure to real fMRI targets and probe reliability to typological proximity.Source: §6 · p. 8
  • Pair-level testing would have made the cross-lingual steering result appear significant (p = 0.0006), whereas the language-level null gives p = 0.155.The authors state that they do not claim to know how often this issue occurs in published work; their own result depended on the permutation choice.Source: §6 · p. 9
  • The brain target is sentence-level, mean-pooled, rank-128 fMRI, while the cross-lingual study uses one model, agreement phenomenon, probe family, layer, and token position.The authors note that syntactically targeted brain representations, other layers or positions, and richer nonlinear interventions could behave differently.Source: §6 · p. 9
  • The authors do not identify why four probes fail; possible explanations include language pretraining share, tokenization, or differences in agreement encoding.Regardless of the cause, transfer from a probe that fails within its own language is uninterpretable, and a cross-language gradient mixes reliability with distance.Source: §6 · p. 9

7 Conclusion

The paper audits brain-alignment and cross-lingual correspondence scores by testing what remains when correspondence is removed and checking whether the measuring instrument works. Across these settings, the authors find substantial score components unrelated to the correspondence and detect no benefit from what survives, within the limits of their operationalizations and designs.

7 Conclusion

  • A permuted or noise target reproduces 83–92% of a brain-alignment score, leaving a brain-specific increment of 5–13% of the reliability ceiling.For four of seventeen languages, the typology gradient is measured through probes at or below chance; the design cannot separate the gradient from measurement error.Source: §7 · p. 9
  • Both interventions work, yet the studies detect no downstream or distance-graded benefit from the remaining correspondence.The conclusion describes these as failures to detect effects of the size the designs resolve, not proofs of absence.Source: §7 · p. 9
  • The findings apply to these operationalizations, and two cases do not establish a general law.The studies use seventeen languages and eight to twenty seeds; the authors say the checks were inexpensive and decisive about what their scores meant.Source: §7 · p. 9

A Cross-lingual study: details

The cross-lingual analysis finds that transfer declines with typological distance, but probe reliability declines along the same axis and is at chance in four languages. The authors test robustness and conclude that this sample cannot separate the instrument’s contribution from the gradient because excluding the unreliable languages also restricts the distance range.

Cross-lingual study: details

  • The distance measure is cosine distance over lang2vec syntax_knn URIEL features, and significance is tested by permutation over language labels rather than 272 pairs.Each language appears in 32 pairs and URIEL distances are phylogenetically structured, so the authors say pairs are not exchangeable; the corrected steering test uses a per-target random-direction baseline averaged over twenty directions.Source: A Cross-lingual study: details · p. 11
  • Transfer declines with distance across seventeen languages (r = −0.66, p=10−4), but the correlation is r = −0.47, p=0.025 among the thirteen above-chance-probe languages.Corrected steering has no significant distance relationship at the language level (r = −0.22, p=0.155); the pair-level value is p=0.0006, and the partial correlation controlling for transfer accuracy is r = +0.11, p=0.455.Source: A Cross-lingual study: details · p. 11
  • Probe-validity thresholds of 0.50, 0.525, 0.55, and 0.60 remove the same four languages (r = −0.468, 156 pairs), while 0.65–0.75 additionally remove German (r = −0.455, 132 pairs).Across 2,380 four-language exclusions, r averaged −0.654 (SD 0.048), and only two attenuated the correlation more than removing the chance-level set did.Source: A Cross-lingual study: details · p. 11
  • Removing the four chance-level languages yields r = −0.47, but it does not estimate the instrument’s share of the gradient because the exclusion truncates typological distance.No distance-matched exclusion exists in this sample; separating reliability from distance requires a sample where they are not collinear.Source: A Cross-lingual study: details · pp. 11–12
  • Four languages have probe-accuracy intervals that exclude the 0.60 threshold, although the estimates are imprecise because the effective test set is about 120 items.The intervals are [0.286, 0.464] for Hindi, [0.344, 0.523] for Urdu, [0.403, 0.582] for Turkish, and [0.407, 0.586] for Arabic.Source: A Cross-lingual study: details · p. 12
  • Hindi’s 0.375 estimate is 2.7 standard errors below chance, but removing Hindi leaves the reported gradients essentially unchanged.The probe uses 17 languages and within-language accuracy averages 0.729 (range 0.375 Hindi to 0.901 Finnish); the study caps training data at 300 pairs per language, with 16 languages trained on 180 pairs and Basque on 163.Source: A Cross-lingual study: details · pp. 11–12

B Brain-alignment study: details

Brain-alignment training raises CKA similarity, but score components remain when the target’s correspondence is destroyed. The reported target-to-target baseline tracks k/n across ranks, and the authors describe the larger-scale replications as bounded rather than high-powered equivalence tests.

Brain-alignment study: details

  • At λ=1, alignment yields BLiMP 74.5 and CKA 0.34, versus LM-only BLiMP 74.1 and CKA 0.10; stronger alignment raises CKA without a BLiMP gain.Across λ=0.1, 1, and 10, CKA rises from 0.18 to 0.34 to 0.38 while BLiMP is 73.6, 74.5, and 73.8, respectively.Source: B Brain-alignment study: details · p. 12
  • Early-, middle-, and late-layer alignment yields BLiMP scores of 74.9, 74.5, and 73.5, each within one point of the 74.1 LM-only score.The 410M and 1.4B comparisons use eight seeds and wider TOST intervals; the authors treat them as bounded replications, not independent high-powered equivalence tests.Source: B Brain-alignment study: details · p. 12
  • The CKA estimator centers both matrices before computing similarity, because uncentered shared means can inflate scores for ablated targets.Paired t-tests use matched seeds, and the reported confidence intervals use the t-distribution; the 160M, 410M, and 1.4B TOST intervals are [−0.4, +1.4], [−0.4, +1.6], and [−1.0, +2.0].Source: B Brain-alignment study: details · p. 12
  • A shuffled target scores CKA(true, shuffled) = 0.204 (sd 0.001), close to the rank-128 baseline of 0.2, while the unaligned model scores 0.095 against the true target.The target-to-target baseline compares targets with each other rather than with a model; a model trained on an ablated target reaches 0.31, above that target’s 0.20 similarity to the true target.Source: B Brain-alignment study: details · pp. 12–13
  • Across ranks 32, 64, 128, and 256, CKA(true, shuffled) is 0.0505, 0.1019, 0.2040, and 0.4088, closely matching k/n values of 0.0510, 0.1021, 0.2041, and 0.4083.The authors conclude that target rank sets this baseline, which can be computed without training a model.Source: B Brain-alignment study: details · p. 13
  • The brain target combines 627 sentences from two Pereira et al. (2018) experiments, retains shared voxels, z-scores them across sentences, and uses the top 128 principal components.Its sentence-count-weighted noise ceiling is 0.54, estimated from participant split-half CKA; raw-voxel split-half CKA is 0.11.Source: B Brain-alignment study: details · p. 13
Loading 2610.03827v4…