NeuroCogMap Reveals Cognitive Organization of Large Language Models

Motivation

• LLMs exhibit broad cognitive-like behaviour, but their cognitive organization remains unclear because they lack explicit modular or task-specific architectures [5].Explaining this organization is important for understanding capabilities, diagnosing systematic failures and testing links to biological cognition.Source: Introduction · p. 1
• Existing behavioural, mechanistic and representation-level approaches do not explain how distributed components form coherent functional units, interact across abstraction levels or fail during pathological behaviour [9] [11] [13].The unresolved gap is a system-level account spanning local mechanisms, broad semantic structure, higher-order capabilities and behavioural control.Source: Introduction · p. 1

Framework

• NeuroCogMap identifies coherent parcels from sparse model features, links them to an interpretable cognitive atlas of functions and capabilities, and organizes them hierarchically from perception through application.This multi-level description parallels principles from human cognitive neuroscience while exploiting LLM states that can be observed, perturbed and replicated at scale [21] [22].Source: Introduction · p. 3

Applications

• NeuroCogMap revealed dissociable disruption patterns across circuits, parcels, capabilities and hierarchy for representational and behavioural-control pathologies, enabling detection and mechanism-guided mitigation.These analyses treated internal cognitive organization as an explanatory level connecting model mechanisms to capabilities and pathology.Source: Introduction · p. 3
• NeuroCogMap parcel activity improved prediction of human cortical responses during naturalistic language comprehension, while latent strategy signatures guided cognitive-model refinements that improved held-out behavioural fit.The strongest neural correspondence involved higher-order cortical systems, without implying biological equivalence.Source: Introduction · p. 3

Functional parcels emerge as stable and interpretable cognitive units in LLMs

• NeuroCogMap identified 270 internally coherent parcels with partially shared activation-pattern organization across Gemma2-2B, Gemma2-9B-IT, and Llama-3.1-8B.The 270-parcel atlas optimized a combined score of 0.771, with mean intra-parcel similarity 0.638 versus inter-parcel similarity 0.433.Source: Figure 2 · p. 4, Functional parcels emerge as stable… · p. 5
• Parcel descriptions predicted held-out activation rankings beyond random, neuron-based, and keyword-only baselines, while targeted interventions produced function-consistent output changes and matched parcels shared cognitive meanings across models.Prediction advantages were significant after Holm correction, and intervention success increased with output-change magnitude.Source: Functional parcels emerge as stable… · p. 5

Parcels organize into cognitive capabilities and hierarchical layers.

• Parcels mapped selectively and causally onto capabilities, which formed a four-layer hierarchy—perception, representation, abstraction, and application—with lower-level supports improving higher-order reasoning.In mixed reasoning tasks, jointly providing contextual retrieval and parametric knowledge-recall supports yielded the strongest performance across model scales.Source: Parcels organize into cognitive… · pp. 5–6

NeuroCogMap reveals representational pathologies in LLMs

• NeuroCogMap analyzed pathology across circuit connectivity, parcel activation, capability recruitment, and hierarchy-level organization, distinguishing representational failures from behavioural-control failures.The framework used structurally constrained functional coupling rather than functional coupling alone to assess directed disruption.Source: NeuroCogMap reveals representational… · p. 6

Hallucination signatures support detection and targeted intervention

• NeuroCogMap detected hallucinations with mean AUROCs of 0.681 in Gemma2-2B and 0.840 in Gemma2-9B-IT, outperforming uncertainty, self-consistency, and representation-probing baselines.Steering under-recruited truthful-response parcels and suppressing over-recruited hallucination parcels improved factual accuracy across MedHallu, NQ-Open, and TruthfulQA.Source: Figure 3 · p. 7, Hallucination signatures support… · p. 8

Hallucination arises through distinct representational routes

• Hallucination followed distinct routes: TruthfulQA failures reflected insufficient higher-order monitoring of misleading representations, whereas NQ-Open failures reflected disrupted coordination among factual-retrieval systems.TruthfulQA hallucinations showed retrieval-dominated, lower-layer activity; NQ-Open hallucinations retained retrieval activity but disrupted factual access, contextual understanding, verification, and answer generation.Source: Hallucination arises through distinct… · pp. 6, 8

Refusal failure reflects a shift from control to procedural execution

• Refusal failure shifted responses from monitoring, evaluation, and negation control toward planning and procedural execution, rather than disrupting harmful-content representation.Successful refusal recruited Legal-Moral Evaluation and safety-related capabilities, whereas failure recruited actionable-response parcels and adaptive-generation capabilities, especially in the Situated Application and Social Interaction layer.Source: Refusal failure reflects a shift from… · pp. 9–10, NeuroCogMap reveals… · p. 9

Refusal-failure signatures support detection and targeted intervention

• Refusal-failure detection achieved mean AUROCs of 0.992 and 0.990 in Gemma2-2B and Gemma2-9B-IT, while steering improved AdvBench refusal accuracy from 38.3% to 98.6% when residual pathology remained.Effects were constrained when baseline refusal was already near ceiling, and additional analyses supported generality beyond the Gemma2 family.Source: Figure 4 · p. 9, Refusal-failure signatures support… · p. 10

NeuroCogMap predicts cortical responses in higher-order networks

• NeuroCogMap parcel activity predicted cortical responses most accurately in higher-order Default, Frontoparietal Control and Salience/Ventral Attention networks.Accuracy was lower in primary sensory systems, consistent with naturalistic story comprehension requiring semantic integration and control.Source: NeuroCogMap predicts cortical… · p. 12

NeuroCogMap captures semantic and representational correspondence with human cortical function

• Top-ranked NeuroCogMap parcels were semantically closer to corresponding human cortical functions than bottom-ranked parcels, with the strongest separation in higher-order systems.The pattern was weaker in Somatomotor and Limbic systems and was replicated by LLM-judge evaluation.Source: NeuroCogMap captures semantic and… · p. 12
• Human–LLM correspondence extended from parcel matching to coordinated systems and selective within-network geometry, including significant alignment in left Default and right Frontoparietal Control networks.RSA yielded r= 0.225, P= 0.047 in left Default and r= 0.774, P= 3.1 × 10−8 in right Frontoparietal Control; right Salience/Ventral Attention showed r= 0.473, P= 0.054.Source: NeuroCogMap captures semantic and… · p. 12

NeuroCogMap predicts neural responses during two-step decision-making

• NeuroCogMap predicted human neural responses during two-step decision-making better than Word2Vec, BERT, Language (Context) and Language (Standard).Mean prediction accuracy was r= 0.355 versus 0.205, 0.339, 0.344 and 0.341, respectively.Source: NeuroCogMap predicts neural responses… · p. 14

NeuroCogMap predicts behavioural fit across cognitive tasks

• NeuroCogMap parcel activity predicted behavioural fit across all five cognitive task families, with strongest associations for episodic memory and weakest for drifting four-armed bandit learning.Correlations were r= 0.598, 0.556, 0.522, 0.508 and 0.213, respectively; parcel patterns reflected task-appropriate versus task-misaligned computations.Source: NeuroCogMap predicts behavioural fit… · p. 14

NeuroCogMap guides refinement of classical cognitive models

• NeuroCogMap-guided latent signatures exposed outcome tracking, uncertainty-aware arbitration and policy switching, enabling interpretable extensions to the classical Dual-systems Model.The extensions targeted omission, dynamic arbitration, history dependence and rule-like switching, providing information beyond behavioural traces alone.Source: NeuroCogMap guides refinement of… · p. 14
• NeuroCogMap-discovered models achieved the lowest held-out AIC on both two-step tasks, outperforming the original and behaviour-only discovered models.AIC was 262.45 versus 268.77 and 268.73 on Two Step Task One, and 465.95 versus 478.29 and 473.53 on Two Step Task Two.Source: Figure 6 · p. 13, NeuroCogMap guides refinement of… · p. 14

Contribution

• NeuroCogMap provides a system-level map linking internal features to functional parcels, cognitive descriptions, capabilities and hierarchy for prediction, perturbation and behavioural interpretation [13].This reframes interpretability as a cognitive-neuroscience-style analysis of artificial systems connecting internal computation to behaviour and pathology.Source: Discussion · p. 15

Pathology and intervention

• Hallucination, social bias, refusal failure and sycophancy reflect separable disruptions in representational coordination or behavioural regulation, enabling mechanism-guided detection and steering.These failures are not uniformly reducible to uncertainty, surface errors or generic hidden-state separability; NeuroCogMap does not claim to solve alignment or safety.Source: Discussion · p. 15

Human alignment

• NeuroCogMap-derived representations predicted human cortical responses and corresponded functionally with higher-order systems for semantic integration, control and context-dependent interpretation.The correspondence is used to relate artificial and biological cognition without assuming mechanistic equivalence.Source: Discussion · p. 15

Model discovery

• Internal model organization revealed candidate latent strategies that, when incorporated into explicit cognitive models, improved accounts of human decision-making.LLMs are treated as manipulable experimental systems for generating hypotheses about cognitive organization and testing them against neural and behavioural data.Source: Discussion · p. 15

Methodological interpretation

• NeuroCogMap uses cognitive neuroscience as a methodological template, defining parcels as distributed, non-spatial feature sets and capabilities as task-grounded constructs rather than human-equivalent modules [9] [13].Parcels may span transformer layers, and correspondence with the brain is assessed at the level of functional organization rather than spatial or mechanistic equivalence.Source: Discussion · p. 15

Limitations and scope

• The framework was evaluated on representative pretrained and instruction-tuned Gemma and LLaMA models, but broader architectures, scales, post-training procedures and multimodal systems remain important tests [24] [25].Multimodal applications must determine whether parcels extend across modalities or require distinct cross-modal organization.Source: Discussion · pp. 15–16

Future directions

• Future work should extend NeuroCogMap from static maps toward dynamic and causal models of internal trajectories, directed interactions and systematic perturbations.This would make cognitive organization experimentally tractable across artificial and biological systems.Source: Discussion · p. 16

Construction of NeuroCogMap

• NeuroCogMap was built by curating 45 capabilities from 5,683 LLM-related papers, extracting SAE activations, clustering response profiles into parcels, annotating them, and organizing capabilities across four cognitive levels.The hierarchy spans perception, representation, abstraction and application, with parcel–capability links based on activation, intervention and semantic evidence.Source: Construction of NeuroCogMap · p. 16, Literature corpus and capability… · p. 16, Parcel–capability mapping and hierarchy · pp. 19–20

Functional atlas construction

• For Gemma2-2B, Gemma2-9B-IT and Llama-3.1-8B, features were filtered, L2-normalized and reduced by truncated SVD, with at least 80% variance retained and k=270 selected.The retained quantiles were 0.5 for Gemma2-2B and 0.8 for Gemma2-9B-IT and Llama-3.1-8B; k=270 maximized the reported score.Source: Functional atlas construction · p. 17

Structural connectome

• The model-level structural connectome aggregates directed forward cross-layer feature relationships between parcel pairs, while capability mappings combine activation, intervention and semantic consistency with signed associations.Positive parcel–capability assignments are separately normalized into a non-negative matrix for detector features and trajectory aggregation.Source: Structural connectome · p. 18, Parcel–capability mapping and hierarchy · pp. 19–20

Cognitive atlas construction

• Parcel annotations used activation-weighted keywords, dataset distributions and high-activation examples, with GPT-OSS-20B assistance and blinded human audits of sampled descriptions, pathology labels and cross-model matches.The resulting atlas treats parcels as non-spatial mesoscopic units in SAE-feature space and uses sentence-level answer-span activations as primary observations.Source: Cognitive atlas construction · pp. 18–19, Parcel–capability mapping and hierarchy · p. 19

Validation of atlas organization

• Atlas validation tested within- versus between-parcel semantic similarity, cross-model parcel correspondence, description-based activation prediction, targeted steering, capability selectivity, hierarchy organization and prerequisite learning.Held-out datasets, random or neuron-based controls, perturbation effects and controlled synthetic training experiments were used across these validation analyses.Source: Validation of atlas organization · pp. 20–21

Pathology analysis, detection and intervention

• Pathology analysis contrasts normative and pathological responses for hallucination, social bias, refusal failure and sycophancy using answer-span parcel trajectories, capability trajectories, functional-connectivity matrices and normative-minus-pathological signatures.Labels follow task-specific datasets and protocols, including BBQ and Sycophancy-Eval.Source: Pathology analysis, detection and… · p. 22
• Pathology detectors used cross-fitted parcel, connectivity, capability and prototype-alignment features with fixed, class-balanced logistic regression and held-out AUROC evaluation.Signatures were selected within training folds using prespecified top-k contrastive statistics, while prototypes, standardization and fitting were also estimated without held-out data. Mitigation enhanced the three most positive or suppressed the three most negative significant parcels, selecting steering strength on validation data.Source: Pathology analysis, detection and… · pp. 22–23

Human cortical prediction and correspondence

• Human cortical correspondence was tested by encoding held-out story-listening fMRI responses from 270 NeuroCogMap parcel predictors and comparing matched cognitive profiles with embedding, judge-based and RSA analyses.The final story, Where There’s Smoke, was held out; baselines used identical encoding procedures but alternative representations. RSA covered Salience/Ventral Attention, Frontoparietal Control and Default networks by hemisphere with Benjamini–Hochberg correction.Source: Human cortical prediction and… · p. 24

Cognitively grounded model discovery

• Cognitively grounded discovery used NeuroCogMap activations to predict trial-wise fMRI and participant-level model fit, then generated and held out extensions of the two-step Dual-systems Model.Discovery used segregated participants, matched behaviour-only and NeuroCogMap-guided agent pipelines, and held-out evaluation; the NeuroCogMap extension added dynamic belief, control, lapse, policy and learning mechanisms derived from parcel- and capability-level summaries.Source: Cognitively grounded model discovery · pp. 25–26

Statistical analysis and implementation details

• Statistical procedures defined analysis units, matched or unequal-variance tests, non-parametric alternatives, multiplicity correction, uncertainty estimates, fixed random seeds and the computational environment.The implementation used Python, PyTorch-compatible activation and intervention hooks, standard scientific Python libraries, and eight NVIDIA RTX A6000 GPUs; exact dependencies and reconstruction instructions were archived publicly.Source: Statistical analysis and… · p. 27

Aggregate category analysis of pathology-related activation differences

• Hallucination and bias produced larger activation differences in belief-related parcels, whereas refusal failure and sycophancy preferentially affected behavioural-control parcels.Capability-level effects were directionally consistent but weaker and nonsignificant for both behavioural-control pathologies, making the separation clearest at parcel granularity.Source: Aggregate category analysis of… · p. 37, Figure 7 · p. 38

Social bias reflects misrouting of socially salient representations

• In BBQ, biased responses showed abnormal coupling between demographic representations and knowledge-retrieval pathways, unlike fair responses’ structured comparison, neutral inference and normative control.This representational pathology was distinct from hallucination because it did not primarily reflect failed truth evaluation or disrupted factual grounding.Source: Social bias reflects misrouting of… · p. 37
• Bias-enhanced connectivity linked Celebrity Knowledge Retrieval to Demographic Comparison (Δ𝑤= −8.68, 𝑃= 1.8 × 10−24) and Wikipedia Fact Retrieval to Demographic Comparison (Δ𝑤= −8.68, 𝑃= 9.3 × 10−35).Fact Extraction Module–Demographic Comparison also increased (Δ𝑤= −6.91, 𝑃= 2.6 × 10−17).Source: Social bias reflects misrouting of… · p. 37
• Fair responses strengthened comparison, enumeration and uncertainty-sensitive circuits, including Factoid Retrieval Module–Item Enumeration (Δ𝑤= 7.80, 𝑃< 10−300).They also strengthened Number-Retrieval Module and Locative Retrieval Module control of Uncertainty Reasoner (Δ𝑤= 7.10, 𝑃< 10−300; Δ𝑤= 6.97, 𝑃= 1.5 × 10−43).Source: Social bias reflects misrouting of… · pp. 37–38
• Fair responses recruited comparison and evaluation parcels, including Duality Coordination (Δ𝑎= 187.24, 𝑃= 2.6 × 10−298), whereas biased responses over-recruited Medical Disease Knowledge (Δ𝑎= −55.61, 𝑃= 3.9 × 10−7).In the disability domain, co-activation of Medical Disease Knowledge and Demographic Comparison treated clinically or socially associated category information as relevant evidence rather than a stereotype-laden cue requiring control.Source: Social bias reflects misrouting of… · p. 38
• Fair responses preferentially recruited evaluative and alignment-related capabilities, including Causal reasoning (Δ𝑐= 19.92, 𝑃= 1.1 × 10−68) and Comprehension and interpretation (Δ𝑐= 13.63, 𝑃= 2.2 × 10−54).Other strengthened capabilities included Induction and inference, Analytical thinking, Ethical reasoning and Alignment.Source: Social bias reflects misrouting of… · pp. 38–39
• At the hierarchy level, bias shifted activation toward Perceptual Access and Attentional Gating, whereas fair responses engaged Abstract Reasoning, Meta-Cognitive Control, Situated Application and Social Interaction.Together, the findings indicate that socially salient category information becomes over-coupled to factual retrieval while higher-order comparison, uncertainty-sensitive reasoning and normative evaluation are insufficiently recruited.Source: Social bias reflects misrouting of… · p. 39

Mechanism-guided detection and mitigation of social bias

• NeuroCogMap achieved mean AUROC 1.000 versus hidden probing 0.999, perplexity 0.883 and attention probing 0.534 on Gemma2-2B, and 0.992 versus 0.991, 0.733 and 0.569 on Gemma2-9B-IT.Across Age, Disability, Gender and Nationality BBQ subdomains, structured parcel-level pathology outperformed generic salience or surprisal measures.Source: Mechanism-guided detection and… · p. 40
• Fairness accuracy increased from 58.3% to 84.5% averaged across BBQ subdomains on Gemma2-2B, whereas Gemma2-9B-IT showed smaller, mixed effects from 80.8% to 81.6%.Intervention was most effective with substantial residual bias and could be neutral or negative when baseline fairness was already high.Source: Mechanism-guided detection and… · p. 40
• Bias altered routing of socially salient information into retrieval and normative-control pathways, while hallucination disrupted factual retrieval, evaluation and cross-system coordination.These structured multilevel signatures were predictive and steerable through parcel intervention, linking systems-level interpretation to mechanism-guided diagnosis and intervention.Source: Mechanism-guided detection and… · p. 40

Sycophancy reflects distributed weakening of independent judgment

• Independent responses recruited normative reasoning, contradiction-sensitive control and self-regulatory decision mechanisms, whereas sycophantic responses relied more on socially compliant or externally guided processing.This distinction separated sycophancy from refusal failure, which reflected a shift into procedural action generation.Source: Sycophancy reflects distributed… · p. 40
• Independent responding showed stronger couplings among Legal-Moral Evaluation, Moral Lesson Detector, Decision-Initiation Module, Contradiction Analysis and Self-Refusal Detection, supporting conflict detection and suppression of premature agreement.Sycophantic responses instead strengthened obedience and self-referential pathways while inhibiting contradiction-sensitive retrieval, including Contradiction Analysis–Factoid Retrieval coupling at Δ𝑤= −0.33.Source: Sycophancy reflects distributed… · pp. 40–41
• Independent responses activated Social Norm Reasoning at Δ𝑎= 13.98 and sycophantic responses activated Uncertainty Clarification Module at Δ𝑎= −12.68, indicating reweighting toward interpersonal alignment.Independent parcels also emphasized structured reasoning, risk evaluation and moral judgment, whereas sycophancy recruited Self-Pride Recognition and other socially receptive modules.Source: Sycophancy reflects distributed… · p. 41
• Independent responses were stronger in Fairness at Δ𝑐= 2.81, Ethical reasoning at Δ𝑐= 2.74 and Verification at Δ𝑐= 1.04, while sycophantic responses retained Common sense reasoning at Δ𝑐= −0.61.The dissociation indicates selective weakening of socially independent judgment and verification rather than a general loss of reasoning.Source: Sycophancy reflects distributed… · p. 41
• Sycophancy showed distributed attenuation across hierarchy levels, beginning with socially relevant attention and extending to socially situated response selection, thereby weakening conflict monitoring, normative evaluation and self-correction.The largest separations occurred in Perceptual Access and Attentional Gating and Situated Application and Social Interaction.Source: Sycophancy reflects distributed… · pp. 41–42

Mechanism-guided detection and mitigation of sycophancy

• AUROC reached 0.685/0.646 on Gemma2-2B and 0.617/0.696 on Gemma2-9B-IT for Answer/Feedback sycophancy detection, outperforming Perplexity, Self-Consistency, and User-Conditioned Attention.NeuroCogMap therefore detected sycophancy better than generic likelihood, sampled-response consistency, or local user-conditioning attention cues.Source: Mechanism-guided detection and… · p. 42
• Sycophancy rates decreased from 28.4% to 26.4% and 59.1% to 57.3% on Gemma2-2B, and from 20.2% to 18.0% and 63.2% to 60.9% on Gemma2-9B-IT across Answer and Feedback.The reductions were modest but consistent, matching the broader and more distributed weakening of independent control identified in preceding analyses.Source: Mechanism-guided detection and… · p. 42
• Refusal failure and sycophancy were characterized as structured output-regulation failures rather than general competence loss, involving released action pathways versus weakened conflict monitoring, normative evaluation, and self-correction.Predictive and steerable control signatures linked systems-level interpretation to mechanism-guided behavioural control.Source: Mechanism-guided detection and… · p. 42

Pathology datasets and representative task patterns

• The pathology analyses separated hallucination and social bias as representational pathologies from refusal failure and sycophancy as behavioural-control pathologies using mechanistically distinct datasets.TruthfulQA and NQ-Open probed misconception suppression and factual retrieval; BBQ tested demographic inference under uncertainty; AdvBench and JBB-Behaviors tested harmful-instruction refusal; Sycophancy-Eval tested factual and evaluative judgment under user pressure.Source: Pathology datasets and representative… · pp. 42–44

Shared evaluation protocol

• Baseline detection used binary normative-versus-pathological labels and the same per-dataset, per-model stratified fivefold cross-validation splits across pathology families.Scalar scores were evaluated with held-out AUROC, while feature-vector baselines used lightweight within-fold classifiers and held-out predicted probabilities.Source: Shared evaluation protocol · p. 44

Hallucination detection baselines

• Hallucination baselines spanned token uncertainty, sequence likelihood, semantic uncertainty, self-consistency, and generic hidden-state probing, without NeuroCogMap’s parcel-, capability-, and circuit-level organization.The comparison followed prior uncertainty-estimation, selective-generation, self-consistency, and hidden-state-separability approaches [35] [36] [33] [34].Source: Hallucination detection baselines · p. 44
• Entropy and length-normalized entropy measured predictive uncertainty, whereas Perplexity assessed response likelihood but could miss fluent high-confidence hallucinations.Semantic Entropy captured disagreement across semantic clusters, while SelfCheckGPT used sampled-response inconsistency but was generation-expensive and could repeat incorrect answers.Source: Hallucination detection baselines · pp. 44–45
• Hidden Probing decoded hallucination labels from generic response-level activations but did not identify the functional parcels, capabilities, or circuit relations contributing to pathology.This distinguished internal-state decodability from NeuroCogMap’s structured mechanistic interpretation.Source: Hallucination detection baselines · p. 45

Refusal-failure detection baselines

• Refusal-failure baselines classify successful refusal against refusal failure or jailbreak success using prompt likelihood, early output distributions, or perturbation sensitivity.These baselines were designed to test whether harmful compliance can be detected without modeling internal control organization.Source: Refusal-failure detection baselines · p. 46
• Logits SVM aggregates early next-token probabilities from one forward pass and classifies the resulting vector with an RBF-SVM.It is computationally efficient because it avoids repeated generations, but treats refusal failure as early-distribution separability rather than structured control breakdown.Source: Refusal-failure detection baselines · p. 46
• SmoothLLM detects refusal failure through response stability across randomized prompt perturbations, contrasting stable benign outputs with less stable adversarial outputs.It measures prompt sensitivity rather than the internal configuration of safety, refusal, and action-generation mechanisms.Source: Refusal-failure detection baselines · p. 46
• Perplexity treats high likelihood scores as evidence that prompts are adversarial or pathological, but can miss fluent harmful instructions and flag rare benign prompts.The baseline is motivated by the distributional unusualness of some jailbreak prompts.Source: Refusal-failure detection baselines · p. 46

Sycophancy detection baselines

• Sycophancy baselines distinguish independent responses from responses shifting toward the user’s stated belief, preference, or self-presentation [96] [33].Because sycophancy has fewer established detectors, the baselines draw from existing sycophancy analyses and consistency-based methods.Source: Sycophancy detection baselines · p. 46
• Self-Consistency [33] [96] measures semantic agreement across repeated samples under user-conditioned prompts, interpreting low consistency or high dispersion as susceptibility to user-guided response shifts.This adapts hallucination-detection consistency measures to sycophancy, where the failure is instability under user pressure rather than factual unsupportedness alone.Source: Sycophancy detection baselines · p. 47
• User-Conditioned Attention uses cue-attention features and a lightweight classifier to test whether sycophancy reflects over-weighting user-belief or user-opinion spans.It probes a local mechanism but does not model broader circuit-level or capability-level control changes.Source: Sycophancy detection baselines · p. 47
• Sycophancy perplexity compares likelihoods of correct, user-suggested incorrect, generated, and neutral responses, but cannot separate social compliance from legitimate evidence-based revision.Features use user-conditioned and neutral-response likelihoods, including absolute and signed differences.Source: Sycophancy detection baselines · p. 47

Baseline setup

• The baselines contrasted unbiased, fair or uncertainty-preserving responses with biased responses to test demographic-attention, hidden-state, and likelihood-asymmetry signals.They were motivated by prior work on stereotypical bias and fairness failures in language models.Source: Social-bias detection baselines · p. 47

Attention Probing

• Attention Probing used answer-token attention to target and counterfactual demographic spans, with A_tar, A_ctr, and A_tar − A_ctr features classified by logistic regression.This tests whether social bias is locally reflected in attention allocation, but not how demographic information affects downstream reasoning or answer selection.Source: Social-bias detection baselines · p. 48

Hidden Probing

• Hidden Probing aggregated middle- and late-layer hidden states into response-level representations and predicted the probability of a biased response.The baseline tests label decodability from internal states, but does not identify responsible functional parcels or capabilities.Source: Social-bias detection baselines · p. 48

Perplexity

• Perplexity compared likelihoods of substantive stereotype-relevant alternatives after removing uncertainty or abstention options only for feature construction.Original fair or unbiased versus biased labels, including uncertainty-preserving responses, were retained for response labeling and AUROC evaluation; the method does not identify the representational mechanism influencing inference.Source: Social-bias detection baselines · p. 48

Comparison with NeuroCogMap

• NeuroCogMap uses coordinated parcel activation, capability recruitment and circuit-connectivity deviations rather than generic uncertainty, instability, attention or hidden-state signals.This makes it a mechanistic comparison point for pathology detection.Source: Comparison with NeuroCogMap · p. 48

Cross-model safety-pathology detection and intervention in Llama-3.1-8B

• In Llama-3.1-8B, NeuroCogMap achieved the strongest plotted hallucination-detection AUROC on NQ-Open (0.691) and TruthfulQA (0.663).It exceeded all listed baselines on both datasets, with the largest margin on TruthfulQA.Source: Figure 10 · p. 49, Cross-model safety-pathology detection… · p. 49
• NeuroCogMap detected refusal failure strongly, reaching AUROC 0.984 on AdvBench and 0.953 on JBB-Behaviors, above Perplexity, Logits SVM and SmoothLLM.These results support stronger cross-model generalization for refusal-failure detection from structured signatures.Source: Figure 10 · p. 49, Cross-model safety-pathology detection… · pp. 49–50
• Parcel intervention improved hallucination and refusal outcomes across Llama-3.1-8B datasets, while sycophancy rates decreased modestly and social-bias effects were heterogeneous.Hallucination accuracy rose from 78.6% to 84.7% on MedHallu, and refusal accuracy rose from 23.2% to 30.2% on JBB-Behaviors.Source: Figure 10 · p. 49, Cross-model safety-pathology detection… · p. 50
• Sycophancy detection was more modest: NeuroCogMap reached AUROC 0.642 on Answer and 0.660 on Feedback, comparable to User-Conditioned Attention.Its values exceeded Perplexity and Self-Consistency, indicating partial but not unique structured-signature decoding.Source: Cross-model safety-pathology detection… · p. 49

Parcel construction is stable across clustering hyperparameters

• Across 16 Gemma2-2B parcellation settings, assignments were highly consistent, with mean similarity 0.786 and within-threshold mean similarity 0.999.Similarity remained especially stable when clustering iterations varied at a fixed SVD-retention threshold.Source: Parcel construction is stable across… · p. 50
• Parcel stability depended on retaining sufficient latent variance: 85% and 90% SVD-retention solutions had mean similarity 1.000, while all observed pairs exceeded permutation-null similarity (P≤0.001).Aggressive 75% retention altered part of the parcel structure, whereas iteration changes had little effect.Source: Parcel construction is stable across… · p. 50

Capability-associated parcels show hierarchical semantic and structural organization

• Across Gemma2-2B and Llama-3.1-8B, higher hierarchy levels showed greater semantic dispersion, whereas lower-level parcels had higher structural degree.Semantic dispersion increased from 0.406 ± 0.011 to 0.466 ± 0.007 in Gemma2-2B, while degree decreased from 62.71 ± 4.10 to 34.84 ± 3.40.Source: Capability-associated parcels show… · pp. 51–52, Figure 11 · p. 53

Synthetic examples for hierarchy-dependency validation

• Synthetic profiles tested in-context, parametric and two-hop reasoning, including mixed conditions combining contextual and profile-derived facts to assess hierarchical support.Each profile contained a name and five attributes, while two-hop examples linked profile pairs sharing an attribute.Source: Synthetic examples for… · pp. 53–54
• Final evaluation supplied available context before each question and counted a prediction correct when temperature-0 generation contained the gold answer string.Generation used top-p=1 and maximum length 50.Source: Synthetic examples for… · p. 54

Psych-101 cognitive-task datasets used in Fig. 6b

• Fig. 6b used five Psych-101 participant-level datasets to predict held-out behavioural fit from Gemma2-2B likelihoods and NeuroCogMap activation vectors.The datasets covered episodic memory, multi-attribute decision-making, Shepard categorization, intertemporal choice and drifting four-armed bandit learning.Source: Psych-101 cognitive-task datasets used… · p. 54
• The episodic-memory task from Popov and Dames combined intentional-versus-incidental word study, arithmetic distraction, semantic judgements and later recall.Model fit reflected likelihoods of observed semantic judgements and recall responses conditioned on preceding study history.Source: Psych-101 cognitive-task datasets used… · p. 54
• The Shepard categorization task from Badham, Sanborn and Maylor required category judgements for geometric objects followed by feedback-based rule learning.Model fit reflected each judgement’s likelihood given previous examples and feedback.Source: Psych-101 cognitive-task datasets used… · p. 55
• The intertemporal-choice task from Ruggeri et al. presented smaller immediate versus larger delayed gains and losses, measuring discounting and framing sensitivity.Model fit reflected the probability assigned to the observed choice given amount, delay and gain/loss frame.Source: Psych-101 cognitive-task datasets used… · p. 55
• The drifting four-armed bandit task from Bahrami and Navajas required sequential choices under changing option values, balancing exploration and exploitation through reward-history updating.Model fit reflected likelihoods of observed arm choices given accumulated rewards.Source: Psych-101 cognitive-task datasets used… · p. 55

Human evaluation of LLM-assisted NeuroCogMap components

• The audit independently assessed LLM-assisted annotation, automated pathology judgements and cross-model matching using blinded files and prespecified human ratings.Three annotators evaluated 60 parcels, 570 responses and 100 matched parcel pairs without seeing hidden automatic labels or scores.Source: Human evaluation of LLM-assisted… · pp. 55–56
• For 60 sampled parcels, descriptions scored 4.24 for specificity, 4.46 for faithfulness and 4.41 for coverage, while overreach was low at 1.71.Ordinal-rating reliability was moderate for specificity, faithfulness and coverage, but lower for overreach.Source: Human evaluation of LLM-assisted… · p. 56
• Majority vote classified 52 parcels as pass, 4 partial, 2 fail and 2 no-majority, yielding a 93.3% pass-or-partial rate.The final categorical decision had low Fleiss’ κ (0.194), partly because ratings concentrated in the pass category.Source: Human evaluation of LLM-assisted… · p. 56
• Across 570 responses, human and automated pathology labels agreed on 426/515 valid coarse classifications (82.7%) and exactly matched on 417/570 items (73.2%).Inter-annotator agreement was substantial (Fleiss’ κ = 0.783).Source: Human evaluation of LLM-assisted… · p. 56
• Refusal-failure labels showed the strongest human–automatic agreement, whereas hallucination had lower exact agreement but high coarse agreement, and sycophancy showed moderate agreement.Exact/coarse agreement was 91.3%/93.8% for refusal failure, 60.7%/94.2% for hallucination and 76.7%/78.8% for sycophancy.Source: Human evaluation of LLM-assisted… · p. 56
• Human raters judged 77/100 Gemma2-2B–Gemma2-9B-IT parcel pairs identical or partially similar (77.0%), supporting most reported correspondences while rejecting a minority.Mean similarity was 2.30 on a 0–3 scale, with high ordinal inter-annotator reliability (ICC = 0.838).Source: Human evaluation of LLM-assisted… · p. 57
• Human similarity ratings correlated strongly with LLM, embedding and combined matching scores, with Spearman ρ values of 0.842, 0.789 and 0.856, respectively.The combined score had the strongest correlation with human ratings.Source: Human evaluation of LLM-assisted… · p. 57

NeuroCogMap supplementary atlas and dataset resources

• Supplementary resources document NeuroCogMap’s cognitive hierarchy, capability taxonomy, construction datasets, Gemma2-2B parcel atlas, cross-model capability mappings and latent-to-parcel assignments.These materials are presented as compact manuscript resources rather than raw data inventories.Source: NeuroCogMap supplementary atlas and… · p. 58, Table 7 · p. 58, Table 8 · p. 58

Supplementary prompt templates

• The templates define structured input-output formats for NeuroCogMap annotation, evaluation, and model-discovery analyses, while omitting internal file paths and execution-specific metadata.The listed return fields match the structured-output fields enforced during implementation.Source: Supplementary prompt templates · p. 75

Cognitive atlas parcel annotation prompts

• Parcel annotations combine ranked keywords, dataset distributions and high-activation question–answer samples to infer each parcel’s primary information-processing role.The prompt requires a concise function name, a 100–200-word description and an account of the parcel’s role in the model.Source: Cognitive atlas parcel annotation… · p. 75

Parcel-function validation and cross-model comparison prompts

• Parcel-function descriptions are scored from 0 to 1 for specificity, example and keyword coverage, mechanistic clarity, neuroscience-style interpretability and avoidance of vague or circular language.The evaluator returns both a numerical score and a brief rationale.Source: Parcel-function validation and… · p. 76
• A redundancy prompt determines whether two parcel descriptions express the same core cognitive function, ignoring superficial wording differences and returning a binary label, similarity score and rationale.It compares function names, descriptions and keywords while focusing on the parcels’ core information-processing roles.Source: Parcel-function validation and… · pp. 76–77
• For each input example, candidate parcels receive estimated activation probabilities and a ranked list justified by their function names, descriptions and keywords.The prompt is designed to predict which NeuroCogMap parcels should be most active.Source: Parcel-function validation and… · p. 77
• Intervention alignment is evaluated by scoring original and post-intervention responses from 0 to 1 and classifying the change as increased, decreased or unchanged.The comparison is made against the specified NeuroCogMap parcel function and includes a rationale.Source: Parcel-function validation and… · p. 77
• Cross-model parcel comparisons assess functional positioning, concept overlap, dataset-distribution similarity and overall semantic relatedness, producing a similarity score and an Identical, Partially Similar or Different label.The comparison uses each parcel’s model, function, keywords and dominant datasets.Source: Parcel-function validation and… · p. 78

• Capability templates specify a capability’s name, definition, cognitive alignment, and manifestation in LLMs.Source: Parcel-to-capability semantic templates · p. 78
• Parcel templates specify each parcel’s identifier, function name, function description, and role in a large model.Source: Parcel-to-capability semantic templates · p. 78
• High similarity indicates semantic consistency between a parcel function and capability definition, whereas low or negative similarity indicates weak or opposing alignment after normalization.Source: Parcel-to-capability semantic templates · p. 78

Hallucination evaluation

• Hallucination prompts assess semantic and factual agreement with reference answers while rejecting unsupported factual content and returning correctness, matched reference, confidence, and rationale.Paraphrases and synonyms are allowed, but unsupported additions invalidate an answer.Source: Pathology evaluation and… · p. 79

Bias evaluation

• BBQ prompts distinguish uncertainty responses from stereotype-aligned bias and non-stereotyped antibias choices, reporting the mapped option, confidence, and rationale.The procedure first checks for an equivalent “cannot be determined” response before mapping other answers to BBQ options.Source: Pathology evaluation and… · p. 79

Jailbreak and refusal-failure evaluation

• Jailbreak prompts evaluate refusal success, harmful content, actionable detail, and overall jailbreak success for harmful requests.The output separately records whether the model refused, included harmful content, and provided detailed instructions supporting harmful intent.Source: Pathology evaluation and… · pp. 79–80

Sycophancy evaluation

• Sycophancy prompts compare neutral and preference-conditioned responses for user-flattering shifts, checking both presentation orders and retaining only consistent paired judgements.The output reports shift presence, directional consistency, confidence, and rationale.Source: Pathology evaluation and… · p. 80

Belief/control classification

• The belief/control classifier assigns NeuroCogMap units to belief-related, control-related, mixed, neutral, or unknown categories with confidence and rationale.Belief-related units concern semantic representation, factual grounding, knowledge integration, belief updating, or representational coherence; control-related units concern inhibition, arbitration, safety regulation, or goal control.Source: Pathology evaluation and… · p. 80

Human cortical description and human–LLM correspondence prompts

• Human cortical parcels are described from Cognitive Atlas terms and signed z-scores, producing concise function names, 100–200-word descriptions, and brief brain-role summaries.The prompt supplies the parcel name and associated term profile as inputs.Source: Human cortical description and… · pp. 80–81
• Human–LLM correspondence is judged by comparing a human cortical parcel with candidate NeuroCogMap parcel descriptions across major cognitive domains.The output includes a 0–1 score, match label, best-matching parcel, and rationale.Source: Human cortical description and… · p. 81

Model-discovery agent prompts

• Behaviour-only and NeuroCogMap-guided discovery share two-step behavioural traces, AIC comparisons, residual summaries, and labels for LLM-favoured regimes.LLM-favoured regimes are defined by lower LLM AIC than the classical model.Source: Model-discovery agent prompts · p. 81
• NeuroCogMap-guided discovery additionally uses parcel and capability activations, cognitive descriptions, and contrasts with regimes where the baseline model matched or outperformed the LLM.These inputs connect behavioural model comparisons to implicated representational systems.Source: Model-discovery agent prompts · p. 81
• The reasoning agent proposes candidate cognitive mechanisms explaining why the LLM simulator fits human behaviour better, combining behavioural evidence with NeuroCogMap evidence when available.Each proposal specifies an expected formal model component and rationale.Source: Model-discovery agent prompts · pp. 81–82
• The model-writing agent converts candidate mechanisms into executable extensions that preserve the task interface, fit participant-level choice data, and use the same search budget.Outputs include code, parameters, mechanism mappings, and compatibility notes for the baseline fitting pipeline.Source: Model-discovery agent prompts · p. 82