PaperlayerView exact revision on arXiv

Source-linked AI summary

Perturbation-based Regional Interpretability through Subtraction Mapping (PRISM): naming-error dissociations in language models and post-stroke aphasia

Xiang Guan, Roger D. Newman-Norlund, Yong Yang, Saeed Ahmadi, Regan Willis, Nadra Salman, Kalil Warren, Srihari Nelakuditi, Chris Rorden, Leonardo Bonilha, Julius Fridriksson

arXiv:2608.12717v1cs.LGcs.CL

TL;DR

Mechanistic interpretability lacks spatially resolved, falsifiable tests of whether model components specialize in distinct cognitive operations. PRISM applies subtraction mapping to perturbed transformers and matched aphasia-lesion analyses, recovering replicated phonemic-favoring dissociations in deep layers and frontal-perisylvian cortex.

  • Problem

    Mechanistic interpretability lacks a shared, externally validated framework for testing whether model components are functionally specialized for distinct cognitive operations.

  • Method

    PRISM applies matched subtraction analyses to perturbed LLaVA layers and 213 aphasia patients, using seed- or patient-level group analyses with spatially ordered TFCE and held-out replication.

  • Results

    PRISM recovered replicated phonemic-favoring dissociations in deep LLM layers and frontal-perisylvian cortex, while the semantic-favoring direction was consistently signed but non-significant.

  • Takeaways & Limitations

    PRISM provides a falsifiable, spatially resolved test of functional-specialization claims in transformer language models.

  • Takeaways & Limitations

    The analysis is confined to one architecture, one behavioral task, and PRISM Stages 1 and 2, with Stage 3 deferred to follow-up work.

Abstract

from arXiv · show

Mechanistic interpretability of large language models lacks spatially resolved, falsifiable tools for testing whether internal components are specialized for distinct cognitive operations. We adapt subtraction analysis, the standard framework of human neuroimaging, from biological brains to perturbed transformers, and apply the same logic to both substrates in parallel. Building on the Brain-LLM Unified Model (BLUM), which showed that layer-perturbed LLaVA-1.6-Vicuna-13B error profiles match the lesion patterns of aphasic patients, we develop PRISM (Perturbation-based Regional Interpretability through Subtraction Mapping). PRISM maps the seven clinical Philadelphia Naming Test categories, subtracts error classes pairwise, and treats each perturbation seed as a subject in a group analysis with threshold-free cluster enhancement along the layer axis. We run a structurally matched analysis on 213 chronic post-stroke aphasia patients using correlation-difference lesion-symptom mapping, and replicate both sides on held-out splits. The designs match in subject dimension (seeds, patients), spatial dimension (layers, atlas-parcellated cortex) and thresholding, but the contrast operator differs: a within-subject error-proportion difference for the LLM, a between-subject correlation difference for the cortex. Both substrates recover a robust phonemic-favoring dissociation, a deep layer cluster and a frontal-perisylvian cortical cluster, both replicating; the semantic-favoring direction is a consistently signed but non-significant trend on both. PRISM thus gives a falsifiable, spatially resolved test of functional-specialization claims in transformer language models. A confirmatory ROI-level intervention (PRISM Stage 3) licensing the strongest causal-mechanism claim is left to subsequent work.

Introduction … Behavioral readout: the Philadelphia Naming Test

PRISM adapts subtraction analysis from cognitive neuroscience to perturbed transformers, using PNT error dissociations to test whether layer clusters specialize for distinct language operations. Applied in parallel to LLaVA-1.6-Vicuna-13B and post-stroke aphasia patients, the framework combines matched behavioral categories, spatial inference, and held-out replication.

  • The interpretability gap: Existing LLM interpretability methods identify encoded or reconstructed information but lack a shared, externally validated framework for testing components’ causal specialization in cognitive operations.The gap concerns behavioral necessity, not merely representation or reconstruction quality.
  • The interpretability gap: Subtraction analysis provides the neuroscience template by identifying brain regions selectively involved in one operation versus another under nonparametric multiple-comparison correction.The approach spans functional neuroimaging and lesion–symptom mapping.
  • Perturbation-induced error maps and PRISM: BLUM showed that perturbed LLaVA-1.6-Vicuna-13B error profiles predicted actual lesions above chance in 67% of picture-naming and 68% of sentence-completion conditions.The corresponding significance levels were p < 10^-53 and p < 10^-68, respectively.
  • Perturbation-induced error maps and PRISM: PRISM treats perturbations as lesion analogues while inheriting subtraction contrasts, cross-replicate inference, and cluster correction from human activation-mapping traditions.Residual connections may prevent a layer perturbation from isolating computation as cleanly as a focal cortical lesion.
  • The PRISM pipeline: For six PNT error categories, PRISM constructs maps across all 40 transformer layers and parallel cortex, computes pairwise subtractions, applies TFCE, and replicates maps on held-out seeds and patients.Layer-order and error-label permutations test whether clustering and separability exceed chance.
  • The PRISM pipeline: The PNT supplies a 175-item, six-error-category taxonomy selected to provide stable behavioral maps and pairwise subtraction signal for anomia.Its categories include semantic, phonemic, mixed, neologism, no response, and unrelated errors.
  • Behavioral readout: the Philadelphia Naming Test: The same seven-category PNT taxonomy is applied to LLM and human responses, enabling comparable subtraction inputs across substrates.Correct responses are held out of PRISM pairwise contrasts.
  • Behavioral readout: the Philadelphia Naming Test: The LLM analysis uses the 158 PNT items named correctly at baseline, because baseline failures cannot reveal perturbation-induced changes in error category.All perturbation conditions are scored on those 158 items; human analysis has no analogous baseline-correct restriction.

Patient lesion data … Human subtraction analysis

The study applies matched subtraction analyses to archival post-stroke aphasia data and layer-perturbed LLaVA-1.6-Vicuna-13B, treating patients and perturbation seeds as subjects. The LLM analysis models perturbation dose and extent before layer-level TFCE inference, while the human analysis compares lesion–error correlations across left-hemisphere atlas regions.

  • Patient lesion data: Of 276 administered patients, N=213 had both a valid PNT administration and a complete JHU ROI lesion-load profile.The analyzed human data were archival, with no new human data collected, and procedures had institutional approval and written informed consent.
  • LLM perturbation grid: LLaVA-1.6-Vicuna-13B lesions were modeled by multiplicative Gaussian perturbations applied to one of 40 successive transformer processing stages.Perturbations were parameterized by layer, noise standard deviation σ, and perturbation density ρ; σ and ρ represented depth and extent rather than position.
  • LLM perturbation grid: The analysis used 80 perturbation seeds split into equally sized discovery and independent validation sets, treating each seed as the subject.Noise magnitude σ and density ρ were aggregated within each seed before group-level spatial inference.
  • LLM subtraction analysis (two-level structure): Within each seed, ordered category-pair contrasts were defined as D_s(ℓ, σ, ρ) = p_s(A∣ℓ, σ, ρ) − p_s(B∣ℓ, σ, ρ), retaining dose-response information.The lower-level model used categorical layer intercepts and continuous within-seed repeated-measures covariates for σ and ρ.
  • LLM subtraction analysis (two-level structure): Across discovery seeds, layer-wise means and one-sample t-statistics were analyzed with TFCE along the layer axis, separately for positive and negative subtraction signs.The seed-level profiles formed the upper-level spatial-extent inference, with NaN-safe estimators.
  • LLM subtraction analysis (two-level structure): The per-seed, per-layer summary was the fitted value at cohort-mean σ and ρ, mathematically equivalent to averaging over the σ×ρ grid.The explicit fit kept σ and ρ in the model, and retained slopes fed a separate dose-response analysis.
  • Human subtraction analysis: Human subtraction was performed across left-hemisphere JHU atlas regions, extending established univariate lesion-symptom mapping to ROI-level cortical contrasts.Atlas-based ROI parcellation was used rather than voxel-wise inference in the context of modest patient sample sizes.
  • Human subtraction analysis: For each ordered error-category pair and ROI, the human analysis contrasted Spearman correlations between patient error proportions and regional lesion loads, using 1,000 patient-level bootstrap resamples for inference.Spearman correlations addressed outliers in heavy-tailed error-proportion distributions, and the procedure was implemented in the SNL_2026_SLM simpleSubtraction viewer.

Out-of-sample replication and Stage 2 permutation validation · Stage 2.5: Per-cluster dose-response analysis · Results

PRISM validates discovery-layer clusters through held-out replication and layer-order permutation, then characterizes each surviving cluster’s perturbation dose-response surface. The analysis applies three pre-registered contrasts to matched LLM and human cohorts while reserving confirmatory ROI intervention for future work.

  • Out-of-sample replication and Stage 2 permutation validation: Held-out validation recomputes LLM layer-axis TFCE maps and human correlation-based maps to test whether discovery-surviving clusters preserve their sign and shape.The LLM uses 40 discovery and 40 validation seeds; the human cohort uses a fixed 50/50 split of 106 discovery and 107 validation patients.
  • Out-of-sample replication and Stage 2 permutation validation: Stage 2 permutes layer labels independently within each seed, preserving marginal contrast distributions while destroying intrinsic layer ordering.The empirical p-value is based on 1,000 permutations whose maximum |TFCE| exceeds the observed value.
  • Out-of-sample replication and Stage 2 permutation validation: A significant permutation result indicates that cluster mass depends on intrinsic layer ordering rather than marginal contrast distributions alone.This test specifically licenses the layer-axis TFCE claim.
  • Out-of-sample replication and Stage 2 permutation validation: Stage 3 is an LLM-only confirmatory ROI perturbation against matched non-significant control clusters and is deferred to a follow-up study.Thus, the reported framework compares LLM and human substrates symmetrically only at Stages 1 and 2.
  • Stage 2.5: Per-cluster dose-response analysis: For each surviving layer cluster, Stage 2.5 averages within-cluster contrasts across perturbation cells and models dose-response effects using perturbation depth σ and extent ρ.These parameters serve as analogs of lesion severity and lesion extent, respectively.
  • Stage 2.5: Per-cluster dose-response analysis: The directional prediction is that β_σ matches the cluster contrast sign, whereas ρ has a less constrained effect because density may deepen or broaden the contrast.A 1,000-iterate cluster bootstrap over seeds supplies 95% confidence intervals for each slope.
  • Stage 2.5: Per-cluster dose-response analysis: Dose-response effects are reported as effect-size estimates and surface shapes, emphasizing direction, magnitude, uncertainty, replication, and shape rather than p-values.The same analysis is repeated on the held-out 40-seed validation cohort to assess surface-shape replication.
  • Results: The study applies three pre-registered confirmatory error contrasts to both substrates: Semantic–Phonemic, Phonemic–Neologism, and Semantic–Neologism.The human analysis uses 213 patients split into 106 discovery and 107 validation participants, while the LLM analysis uses 80 seeds split into 40 and 40.

Per-category univariate signal on both substrates

Before pairwise subtraction, every PNT error category shows category-specific signal in both LLM layers and human cortical ROIs. Subtraction then reveals a replicated asymmetry: phonemic-favoring effects are robust, whereas semantic-favoring effects remain consistently signed but non-significant.

  • LLM layer profiles: Neologism peaks in mid-network layers, Mixed accumulates gradually, Unrelated peaks mid-network before declining, and NoResponse begins high before declining.
  • Per-category univariate signal: Univariate maps show category-specific signal for every non-Correct PNT error category across both LLM layers and human cortical ROIs.The LLM analysis averages across 40 discovery seeds, while the cortical analysis covers N=213 patients without covariate adjustment.
  • Human cortical contrasts: Human subtraction maps show a robust left frontal-perisylvian Phonemic > Semantic cluster, while the Semantic > Phonemic direction is consistently signed but non-significant.Phonemic- and neologism-favoring contrasts converge on postcentral, precentral, supramarginal, and superior longitudinal fasciculus regions; semantic-favoring panels are weaker and more diffuse.
  • LLM layer contrasts: The LLM primary contrast yields a robust Phonemic > Semantic cluster at layers 22–31 on discovery and 23–33 on validation, with no surviving Semantic > Phonemic cluster.The discovery cluster has a TFCE peak at layer 22, its largest mean difference near layer 29, and Stage-2 layer-permutation p<0.001.
  • Cross-substrate asymmetry: Across confirmatory contrasts, surviving LLM clusters favor Phonemic or Neologism, while the semantic-favoring direction remains a consistently signed but sub-threshold trend matching the human asymmetry.Phonemic > Neologism occurs at layers 24–31; Neologism > Semantic occurs at layers 16–17 and 19–20.

Out-of-sample replication on both substrates … Importing lesion-symptom mapping into LLM interpretability

PRISM’s phonemic-favoring dissociation replicated across held-out LLM seeds and aphasia patients, while dose-response analyses qualified the perturbation-as-lesion analogy. Across substrates, matched spatial-inferential machinery recovered an asymmetric result that strengthens falsifiability beyond prior interpretability approaches.

  • Out-of-sample replication on both substrates: Held-out LLM validation reproduced discovery layer clusters, with the primary Phonemic > Semantic cluster shifting only from layers 22–31 to layers 23–33 while preserving sign and location.The Neologism cluster likewise preserved sign and location, spanning layers 24–33 on validation versus 24–31 on discovery.
  • Out-of-sample replication on both substrates: All 15 observed pair-level clusters cleared the empirical p< 0.001 threshold against an independently permuted-layer null.The result indicates that cluster mass was licensed by the intrinsic ordering of layers.
  • Out-of-sample replication on both substrates: 714 (74%) of 960 human pair–ROI tests matched sign across discovery and validation, while 60 (6.3%) met the strict replication criterion.Strictly replicating primary-contrast ROIs were exclusively Phonemic > Semantic.
  • Dose-response structure of the recovered layer clusters: Recovered phonemic-favoring layer clusters showed dose-response signatures for perturbation depth σ and extent ρ, the LLM analogs of lesion severity and extent.The primary Semantic-vs-Phonemic cluster was defined at layers 22–31, with discovery-seed OLS slopes reported for σ and ρ.
  • Dose-response structure of the recovered layer clusters: Across all three clusters, mean |D| peaked along a diagonal σ×ρ ridge where perturbation depth and extent traded off inversely, rather than increasing monotonically.The directional slopes summarize average grid dependence but do not capture the full surface.
  • Dose-response structure of the recovered layer clusters: The bilinear-plus-interaction OLS explained only R2 ≈0.18–0.26 of each dose-response surface, and both the limited fit and saturating shape replicated in validation.These findings constrain the perturbation-as-lesion analogy at very high perturbation levels.
  • Summary of principal findings: PRISM applied subtraction analysis to a 13-billion-parameter vision-language transformer and 213 chronic-aphasia patients using parallel subject, spatial, and TFCE-style thresholding dimensions.The LLM used seeds and transformer layers; the human analysis used patients and atlas-parcellated cortex.
  • Convergence with the human LSM maps: All three contrasts recovered a phonemic-favoring direction that survived both substrates’ replication standards, strengthening falsifiability beyond BLUM and benchmark-based interpretability methods.The study demonstrates shared conceptual machinery without claiming detailed cross-substrate map alignment; unlike dominant methods, it evaluates behaviorally defined causal necessity.

Dose-response evidence that the perturbation behaves as a graded lesion … Data availability

PRISM’s dose-response scaling supports a graded-lesion interpretation while limiting layer-cluster claims to behaviorally grounded, coarse-grained surrogates. The study reports defined replication limits, proposes confirmatory and cross-architecture extensions, and provides ethics and data-access details.

  • Dose-response evidence that the perturbation behaves as a graded lesion: The recovered cluster contrast scales with perturbation dose rather than remaining flat, supporting a functioning lesion analog; saturation at extreme σ and ρ limits category-specific interpretation.At very high σ and ρ, all error categories emerge together and category-differentiating signal degrades.
  • What layer-cluster claims license: PRISM identifies layer ranges necessary during the perturbed forward pass, not computationally sufficient standalone units, because residual connections couple layer outputs to preceding layers.The method is behaviorally grounded and coarse-grained by design, complementing component-level interpretability.
  • What layer-cluster claims license: The identified perturbation conditions can serve as layer-cluster-level computational surrogates for patient-specific digital twins, enabling in-silico screening of trial designs and individualized rehabilitation strategies.These surrogates are motivated by behavioral profile matching within the present architecture.
  • Limitations: The analysis is limited to one architecture, one behavioral task, and PRISM Stages 1 and 2, leaving replication across architectures, scales, tasks, and aphasia subtypes unresolved.The listed tests include pure-text and encoder-decoder models, additional parameter scales, clinical tasks, and multiple aphasia subtypes.
  • Limitations: The perturbation parameters have loose clinical analogs: σ indexes damage depth or completeness, while ρ indexes lesion extent; these are framed as analogies rather than equivalences.σ controls multiplicative perturbation strength, and ρ controls the fraction of layer weights perturbed.
  • Future directions: The immediate extension is PRISM Stage 3: confirmatory ROI perturbation of the phonemic-favoring cluster against matched non-significant control clusters on the LLM side.Cross-architecture and cross-task replication are proposed beyond Stage 3.
  • Conclusions: Across matched analyses of a perturbed 13-billion-parameter transformer and 213 chronic-aphasia patients, the main contrasts and Neologism checks recover robust phonemic-favoring maps in both substrates.The conclusion describes matched subject, spatial, and TFCE machinery across the two substrates.
  • Ethics statement: Human procedures received University of South Carolina IRB approval Pro00053559 with written informed consent, and no new human data were collected.The patient data came from the Aphasia Lab’s archival database.

Code availability … S2.1 Per-seed per-layer contrast

The supplementary materials document PRISM’s implementation, availability, interests, perturbation grid, behavioral readout, and seed-as-subject layer-axis contrast. The analysis uses seven category proportions across perturbation conditions, with independent discovery and validation seeds and dose parameters collapsed into per-seed layer profiles.

  • Code availability: The full PRISM pipeline, cortical analyses, replication code, figure scripts, and LLM-derived data are deposited in a Zenodo record under Apache License 2.0.The deposit covers three-stage LLM analysis, cortical ROI-correlation and split-replication code, and figure-generation scripts.
  • Declaration of interests: J.F. reports ownership interests in NXTLLM, LLC, and ALLT.AI, LLC, and J.F. and R.D.N.-N. are inventors on a related provisional patent application.The cited interests concern BLUM-related patents and methods described in the work.
  • PRISM Supplementary Materials: The supplementary document provides formal definitions, parameters, and implementation details corresponding to the main-text Methods section.Its section numbering parallels the main text where useful.
  • S1. Perturbation grid and behavioral readout: Formal and Nonword errors are merged into Phonemic, yielding seven categories aligned with the BLUM clinical taxonomy; Correct is excluded from pairwise contrasts.Zero-scored cells are treated as missing and propagated NaN-safely, while contrasts against Correct are excluded because they re-express overall error rate.
  • S1. Perturbation grid and behavioral readout: The discovery dataset contains 40 seeds × 4,000 cells × 7 category proportions, with an independent 40-seed validation dataset and an 80-seed equal-size split.The validation seeds are drawn from the same distribution as the discovery seeds.
  • S2.1 Per-seed per-layer contrast: For each ordered category pair, perturbation-dose conditions are collapsed within each discovery seed to produce one profile along the layer axis, treating seeds as subjects.σ and ρ represent perturbation depth and extent rather than spatial coordinates, so their grid positions are not used for cluster correction.

S2.2 Group-level mean, standard error, and t-statistic across seeds … S3. Cortical LSM pipeline

The analysis aggregates seed-level layer contrasts with capped t-statistics, signed TFCE, cluster extraction, permutation validation, and a repeated-measures sensitivity analysis. The cortical pipeline applies analogous subtraction-based inference to atlas ROIs using patient-level lesion-symptom correlations and bootstrap confidence intervals.

  • S2.2 Group-level mean, standard error, and t-statistic across seeds: Seed-level layer contrasts use NaN-safe means, Bessel-corrected standard errors, one-sample t-statistics, and a hard t-cap of 50.The cap prevents numerical instability when a layer has effectively zero variance across seeds.
  • S2.3 Signed one-dimensional TFCE along the layer axis: Signed one-dimensional TFCE is applied separately to positive and negative t-statistics along the ordered layer axis and then recombined with sign.The operator sums extent-weighted contributions across 200 evenly spaced positive thresholds using E = 0.5 and H = 2.0.
  • S2.4 Layer-cluster extraction: Layer clusters survive the 90th-percentile signed-TFCE threshold only when contiguous runs span at least 2 layers, and are ranked by total TFCE mass.Each cluster is summarized by its start and end layers, extent, peak layer, peak t-value, and total TFCE mass.
  • S2.5 Stage 2 layer-order permutation validation: Layer-order validity is tested by independently permuting layer labels within each seed and recomputing group t-statistics and signed TFCE across 1,000 permutations.The permutation null preserves each seed’s per-cell marginal distribution while destroying layer ordering and records maximum signed |TFCE| by direction.
  • S2.6 Repeated-measures variant of the per-seed summary: The repeated-measures sensitivity variant models layer dummies with σ and ρ as fixed-effect covariates within each seed before applying identical group-level TFCE machinery.This variant recovers the Phonemic > Semantic discovery cluster at layers 22–31, matching simple averaging.
  • S3. Cortical LSM pipeline: The cortical pipeline uses left-hemisphere JHU atlas ROIs, patient-level PNT error proportions, and lesion-load profiles for lesion-symptom mapping.The inputs are defined on the 276-patient cohort, while later bootstrap inference uses 213 patients.
  • S3. Cortical LSM pipeline: For each ordered error-category pair and ROI, cortical subtraction is the difference between Spearman correlations of error proportions with regional lesion load.Positive values indicate stronger association with category A than B; negative values indicate the reverse.
  • S3. Cortical LSM pipeline: Cortical robustness is assessed with patient-level bootstrap 95% confidence intervals from 1,000 resamples, while ROI-level inference parallels the LLM subtraction framework.An ROI is inferentially robust when its interval excludes zero in the same direction as its point estimate.

S4. Out-of-sample replication

Out-of-sample analyses replicated the primary phonemic-favoring dissociation in both the perturbed LLM and post-stroke aphasia cohort. The LLM replication preserved the deep-layer cluster, while the human replication identified strictly concordant phonemic-over-semantic cortical ROIs.

  • LLM side: The LLM replication independently refit the seed-as-subject pipeline on 40 validation seeds using the same summaries, statistics, TFCE, and cluster extraction.Each seed’s summary averaged σ and ρ within seed before group-level analysis.
  • LLM side: Layers 23–33 reproduced the Semantic-versus-Phonemic Phonemic > Semantic cluster, overlapping the discovery cluster at layers 22–31 with matching direction and sign.Replication required overlap in the same direction and matching sign of D̄ at overlapping layers.
  • Human side: The human analysis split 213 patients into 106 discovery and 107 validation patients, recomputing ROI-wise correlation differences independently with 1,000-resample bootstrap intervals.The split used seed 42 and a 50/50 design.
  • Human side: Strictly replicating cortical ROIs occurred exclusively in the Phonemic > Semantic direction: PoCG_L and PrCG_L.Strict replication required matching signs between discovery and validation correlation differences, with bootstrap 95% confidence intervals used in the analysis.

S5. Formal inference and multiple-comparison correction

Inference combines layer-axis permutation evidence with held-out validation for LLM clusters, while human inference uses patient-level bootstrap confidence intervals and split-sample replication. Multiple comparisons are controlled across the 15 LLM contrasts, with optional Bonferroni correction across 64 left-hemisphere ROIs for human analyses.

  • LLM inference: LLM cluster inference combines Stage 2 layer-axis TFCE p-values against the destroyed-ordering null with sign and shape preservation on 40 held-out seeds.The discovery run found empirical p< 0.001 across all observed clusters.
  • Multiple-comparison correction: Multiple-comparison correction across 15 pairwise LLM contrasts uses layer-permutation p-values directly, with Bonferroni α′ = 0.05/15.
  • Human inference: Human inference uses the patient-level bootstrap 95% CI on d_r as the per-(pair, ROI) test statistic, with sign preservation and CI overlap across 50/50 splits as replication metrics.No additional ROI-level permutation null is computed.
  • Multiple-comparison correction: For family-wise correction, tabulated two-sided Spearman p-values can support Bonferroni correction at α′ = 0.05/64 across 64 LH ROIs.

S6. Implementation and reproducibility

The reproducible Python pipeline separates discovery, out-of-sample evaluation, and visualization, while implementing explicit TFCE and NaN-safe statistics. Seed-summary CSVs use a documented schema, and the end-to-end all-pairs sweep runs on commodity hardware in under 30 minutes.

  • Pipeline organization: The full pipeline is implemented in Python and organized into three stages: discovery, out-of-sample evaluation, and visualization.It uses numpy, pandas, matplotlib, and scipy.
  • Statistical implementation: The 3D TFCE operator is implemented from first principles with explicit threshold enumeration, connectivity, and d_h-spacing conventions.This design supports planned layer-order and label-permutation testing in stage 2.
  • Statistical implementation: NaN-safe implementations of the mean, variance, and t-statistic prevent spurious all-NaN warnings under sparse seed coverage.
  • Data and runtime: The discovery and validation seed-summary CSVs document a shared schema covering seed, layer, noise, error proportions, correctness, and seven naming-error categories.The schema fields are Seed, Layer, NoiseStd, Percent, Correct, Semantic, Unrelated, Formal, Nonword, Mixed, Neologism, and NoResponse.
  • Data and runtime: Under 30 minutes is sufficient to run the build, evaluation, and plotting scripts end-to-end for the full all-pairs sweep on commodity hardware.

S7. Per-category univariate analyses (pre-subtraction marginals on both substrates)

Per-category univariate maps establish that each PNT error category has spatial or layer-wise signal before subtraction. Phonemic and Semantic maps partially overlap across substrates, motivating subtraction to isolate differential signals.

  • S7. Per-category univariate analyses: Per-category maps were computed before subtraction on both substrates to verify that each PNT error category carries its own spatial or layer-wise signal.Cortex maps used lesion-load correlations, whereas LLM maps used per-seed, per-layer mean error proportions.
  • S7.1 Cortex side: 213 patients and 64 LH JHU atlas ROIs defined the cortex-side analysis, without age, sex, lesion-volume, or other covariate adjustment.The analysis computed two-sided Spearman correlations between per-patient category proportions and ROI lesion loads.
  • S7.1 Cortex side: 384 tests yield a Bonferroni threshold of α′ = 0.05/384 ≈1.3 × 10−4, or α′ = 0.05/64 ≈7.8 × 10−4 when categories are treated as independent families.Table S5 reports the top five ROIs per category by |ρA,r| under both correction levels.
  • S7.3 Cross-substrate reading: Phonemic peaks at layer 29 with a half-max span of L14–L39, while Semantic peaks at layer 39 with span L17–L39.Both LLM profiles are broad and partially overlapping, with Phonemic described as late-mid and Semantic as late.
  • S7.3 Cross-substrate reading: Phonemic and Semantic cortical maps overlap in perisylvian territory, with SLF_L, SMG_L, PoCG_L, and PSTG_L among the top ROIs for both categories.The shared loading is the basis for interpreting subtraction as a way to isolate differential rather than common category structure.

Supplementary Tables

Supplementary analyses show that the phonemic-favoring dissociation is the most reproducible cross-substrate finding, with matched LLM layer and cortical ROI patterns that survive validation. Semantic-favoring effects remain directionally consistent but weaker, while dose-response and category-specific profiles support the subtraction interpretation.

  • LLM layer inference: Phonemic-favoring contrasts produced nearly identical TFCE layer clusters across discovery and validation, whereas semantic-favoring contrasts produced no surviving cluster on either half.Stage 2 layer-ordering tests nevertheless recovered some structure for semantic-favoring contrasts, without extractable clusters.
  • Human cortex ROI inference: Four cortical ROIs—PoCG_L, PrCG_L, SLF_L, and SMG_L—had bootstrap 95% CIs excluding zero for the primary Phonemic > Semantic direction.The same frontal-parietal/perisylvian territory was independently recovered by the Neologism > Semantic cross-check.
  • Human cortex ROI inference: Semantic > Phonemic effects were consistently negative in left posterior temporo-occipital and subcortical regions, but all bootstrap CIs crossed zero at N=213.The passage attributes this asymmetry to broader spatial distribution and smaller per-region magnitude.
  • Cross-substrate pairwise contrasts: Phonemic and Unrelated contrasts isolated frontal-perisylvian PoCG/PrCG/SLF clusters, while Semantic and Neologism contrasts isolated posterior temporo-occipital MOG/IOG/Cu clusters.This cross-substrate pattern was strongest when specialized real-word error classes were contrasted against less-targeted baselines.
  • Dose-response analysis: Across confirmatory contrasts, phonemic-favoring dose-response slopes matched the cluster contrast sign and replicated within 10% across seed splits.Exploratory deviations were largest for NoResponse-baseline contrasts, where total error-rate effects dominated.
  • Category-specific profiles: Every non-Correct PNT category showed category-specific cortical signal and a coherent peaked LLM layer profile, although Mixed lacked a dedicated cortical correlate.Phonemic and Semantic profiles overlapped substantially, with subtraction isolating the differential residual in cortex and layers.

Supplementary Figures

The supplementary figures provide per-category and all-pairs subtraction maps for both substrates, alongside permutation-significance and dose-parameter visualizations. They show that all observed Stage-2 clusters were significant, while TFCE-surviving slabs appeared only for the three directional contrasts favoring Phonemic or Neologism.

  • Per-category maps: Per-category maps show LLM layer-wise error proportions with seed-level 95% CIs and cortical ROI-wise Spearman correlations between error proportions and lesion load.The cortical maps use the 64-region JHU atlas rendered on the MNI152 T1 template; the LLM maps average σ and ρ within seed.
  • All-pairs contrasts: The supplementary all-pairs maps display all 15 ordered contrasts across the six non-Correct categories for both layer-axis profiles and cortical correlation differences.Pair naming and ordering match across substrates, enabling direct pair-by-pair visual comparison.
  • Permutation significance: Every observed Stage-2 cluster across the 15 pairwise contrasts cleared empirical p<0.001 under the 1,000-permutation layer test.The observed cluster mass was at the upper extreme of the layer-permuted null in every tested direction.
  • Stage-2 visualizations: TFCE-surviving slabs appeared for Phonemic > Semantic, Phonemic > Neologism, and Neologism > Semantic, but not for the three reverse directions.The 3D renderings span the full σ×ρ grid, while inference is performed only along the layer axis.
Loading 2608.12717v1…