PaperlayerView exact revision on arXiv

Source-linked AI summary

Modular Cognitive Architecture Emerges in Large Language Models

Pengrui Han, Jacob Andreas, Evelina Fedorenko, Andrea Gregor de Varda

arXiv:2608.13567v1cs.AIcs.CLcs.LG

TL;DR

The paper asks whether modular organization is a general property of intelligence or an accident of biological evolution. Using circuit analyses across 46 tasks and four cognitive domains, it finds that LLM neuron organization mirrors human brain networks, with overlapping circuits for same-network tasks and distinct circuits for different networks. The authors conclude that modularity may be a general principle of intelligent systems.

  • Problem

    The study asks whether the modular organization of human cognition is fundamental to intelligence or specific to biological evolution.

  • Method

    The authors analyze neuron circuits across N=46 tasks spanning four cognitive domains and compare their organization with human functional networks.

  • Results

    LLMs develop modular organization: same-network tasks recruit substantially overlapping neurons, whereas different-network tasks rely on largely distinct neuron sets.

  • Takeaways & Limitations

    The convergent organization suggests that modularity may constitute a general principle of how intelligent systems are organized.

  • Takeaways & Limitations

    The transformer networks studied do not face the metabolic pressure that may contribute to specialization in biological brains.

Abstract

from arXiv · show

The human brain exhibits a striking degree of functional specialization, with distinct networks supporting language, formal reasoning, reasoning about other minds, and reasoning about the physical world. Is this modular organization a fundamental principle of how intelligent systems must be built, or an evolutionary accident specific to biological brains? Here, we test whether a similar organization emerges in Large Language Models--another class of intelligent systems created through a very different optimization process. Using circuit analyses across N=46 tasks spanning four cognitive domains (language, formal reasoning, social reasoning, physical reasoning), we find that LLMs develop a modular architecture that mirrors the human brain: tasks drawing on the same network in humans recruit overlapping neurons in LLMs, whereas tasks drawing on different networks recruit distinct neurons. The convergent emergence of modularity in brains and neural networks suggests that it may be a fundamental property of intelligent systems.

Introduction

The paper asks whether modular functional specialization is fundamental to intelligence rather than an accident of biological evolution. It tests whether LLMs develop human-like organization across language and three reasoning domains.

  • Motivation: Human brains show remarkably consistent modular organization despite variation in lateralization and hemisphere number.Virtually every neurotypical brain exhibits the same broad functional modularity.
  • Motivation: Modularity is a candidate principle of intelligence because it independently emerges in systems facing similar computational demands.LLMs provide a comparison system optimized without the biological constraints of evolution and metabolism.
  • Research question: LLMs exhibit human-like behavior and performance across language, reasoning, and social-inference tasks, making them suitable systems for testing this question.The study asks whether their internal organization resembles the modular organization found in human brains.
  • Cognitive domains: Human neuroscience distinguishes language, formal reasoning, social reasoning, and physical reasoning as partly distinct cognitive systems.These domains are associated with language, Multiple Demand, Theory of Mind, and Intuitive Physics networks, respectively.
  • Study design: The study presents a comprehensive investigation of fine-grained modular organization across N=46 tasks spanning four cognitive domains.The domains are language processing, formal reasoning, physical reasoning, and social reasoning, each grounded in a distinct functional brain network.
  • Study design: The central finding is that LLM circuits mirror human network organization: same-network tasks overlap, whereas different-network tasks recruit largely distinct neurons.This comparison targets modularity across both language-versus-reasoning and distinct reasoning domains.

Results

Across six capable LLMs, task-selective neuron circuits formed modular groupings aligned with cognitive domains and human functional networks. Causal ablations confirmed that these circuits preferentially support tasks within their own domains.

  • Circuit identification: Attribution patching estimates individual neurons’ task contributions by comparing original problems with alternative versions and then identifies the top 0.1% neurons per task.The study validated the attribution results with targeted ablations across the task battery.
  • Experimental scope: The analysis covered 46 tasks across four cognitive domains in six state-of-the-art language models ranging from 24B to 123B parameters.Average performance ranged from 81% in physical reasoning to 85% in language tasks.
  • Modular organization: Within-domain neuron overlap was 12.9% versus 3.0% across domains, exceeding cross-domain overlap more than fourfold.The pattern held within each cognitive domain and in each model; overlap structures were consistent across models with mean pairwise Kendall’s τ = 0.70 ± 0.06.
  • Modular organization: Hierarchical clustering recovered the four predefined cognitive domains with adjusted Rand index = 0.78, p < 0.0001.This provided a bottom-up test of whether neuron-overlap structure matched boundaries defined by cognitive neuroscience.
  • Network depth: Domain-relevant neurons were concentrated in mid-to-late layers but largely non-overlapping, while language neurons also appeared substantially in early layers.The authors suggest that linguistic computations parse inputs early, creating representations for downstream reasoning systems.
  • Robustness: The modular structure did not emerge in low-performing GPT-2 and remained after controls for prompt semantic similarity.These results indicate that the overlap structure captures task computations beyond semantic content and depends on the model solving the tasks.

Discussion

LLMs develop a modular organization resembling human brain specialization, despite lacking biologically meaningful metabolic pressure. This modularity may arise because systems solving diverse problems must limit interference between computations.

  • LLMs develop the same broad modular organization that characterizes the human brain, including language, formal reasoning, social reasoning, and physical reasoning.
  • Tasks within a domain recruit substantially overlapping neurons, whereas tasks supported by different networks rely on largely distinct neuron sets.
  • Modularity emerges despite transformers lacking biologically meaningful metabolic-cost variation and a loss-function penalty on active neurons.
  • Separating computational subpopulations can protect representations from interference and allow learning in one domain without overwriting knowledge in another.
  • The convergent emergence of modularity suggests that pressure to prevent interference may suffice to produce modularity in intelligent systems solving diverse problems.
  • LLMs may help neuroscience generate hypotheses about brain networks and test mechanistic information-flow questions that are difficult to resolve with neuroimaging.

machines. Annual Review of Neuroscience, 47(1), 277-301.

The cited related work includes studies of task representations in neural networks and decoupled functional modules in large language models.

  • Yang et al. (2019) studied task representations in neural networks trained to perform many cognitive tasks.
  • The cited studies were published in Nature Neuroscience and the AAAI Conference on Artificial Intelligence, respectively.
  • Yu et al. (2026) reported work on discovering decoupled functional modules in large language models.

1. Methods

The study analyzes 46 tasks across four cognitive domains in six instruction-tuned LLMs, using minimally different input pairs and neuron-level attribution patching to identify task-critical circuits. Tasks are assigned to domains based on hypothesized correspondences with human functional networks.

  • Benchmark design: 46 tasks span language, multiple-demand reasoning, physical reasoning, and social reasoning domains.The benchmark includes 8 language tasks, 20 multiple-demand reasoning tasks, 9 physical reasoning tasks, and 9 social reasoning tasks.
  • Benchmark design: Minimally different original and alternative inputs change the correct answer while holding surface features or task context as constant as possible.Examples include changing a plural noun, operand, operator, physical context, or social context so that the opposite answer becomes correct.
  • Circuit analysis: Neurons are localized by measuring how much restoring their original activation in an alternative forward pass recovers the model’s correct behavior.The method is used to identify neurons most critical for each task while approximating more expensive causal analyses.
  • Evaluation criteria: Analysis retains items only when the model answers both original and alternative versions correctly, requiring at least 60% both-correct performance per task.The stated chance level is 25%.
  • Domain assignment: Domain assignments are based on human-brain functional-network characterizations, although many included tasks lack direct neural-response validation.Some mappings therefore rely on broad characterizations, such as assigning social reasoning to the Theory of Mind network.
  • Models: Six publicly weighted instruction-tuned LLMs from four families span 24 to 123 billion parameters, enabling comparisons across architectures and scale.The selection covers approximately a five-fold parameter range.
  • Circuit analysis: Attribution patching estimates each neuron’s causal task importance from its activation difference between paired inputs and the gradient of the task metric with respect to its activation.The analysis focuses on hidden neurons in the feed-forward intermediate layers of transformer blocks.

2. Data description

The study assembles 46 minimal-pair tasks across language, formal reasoning, physical reasoning, and social reasoning. Each corruption changes a targeted feature so that the correct continuation or answer flips.

  • Task domains: The dataset spans four cognitive domains, including language, formal reasoning, physical reasoning, and social reasoning.Language tasks cover morphological, syntactic, and lexical-semantic processing; formal tasks include arithmetic, logic, code execution, and algorithmic reasoning.
  • Task sources: Six language tasks come from BLiMP, two additional language tasks come from Hanna et al. (2026), three tasks come from Reasoning Gym, and the remaining tasks are generated by the authors.The supplied passages explicitly identify the BLiMP and Hanna et al. sources and state that Eq, NumSeq, and NumSort come from Reasoning Gym.
  • Task construction: All tasks use minimal pairs whose original and alternative inputs differ in a single feature that flips the correct continuation.The changed feature may be syntactic, lexical, numerical, logical, or task-specific.
  • Language datasets: Language probes test anaphor resolution, determiner–noun and subject–verb agreement, negative-polarity licensing, hypernymy, and morphological generalization.The tasks manipulate antecedent gender, number agreement, negation, category membership, or singular/plural context.
  • Formal reasoning datasets: Formal reasoning probes cover symbolic and verbal arithmetic, linear equations, syllogistic and propositional logic, and code comprehension.The probes alter operands, constants, quantifiers, question polarity, operators, or code tokens to change the answer.

3. Per-task accuracy across models

Across six large instruction-tuned models, performance was high and broadly balanced across the 46 tasks and four domains. Most tasks met the study’s 60% both-correct inclusion threshold for at least three models.

  • Accuracy measure: Both-correct accuracy measures the fraction of minimal pairs where the model favors the matching continuation for both original and alternative inputs.Chance performance is 0.25.
  • Domain comparison: Per-domain mean accuracy was balanced: Language 0.84 ± 0.06, Formal 0.85 ± 0.08, Physics 0.81 ± 0.08, and Social 0.83 ± 0.07.These means summarize both-correct accuracy across the four cognitive domains.
  • Model coverage: Per-model task coverage ranged from 35 of 46 tasks for OLMo-2-32B to 46 of 46 for Qwen2.5-72B.All other tasks had broad cross-model coverage apart from Mul3-Vrb.
  • Figure interpretation: Supplementary Figure 1 places models in rows and tasks in columns, groups tasks by color-coded domain, and encodes accuracy with cell color.Domains are Language, Formal, Physics, and Social; cell colors range from 50% to 100%, with values below 50% clipped at the floor.

4. Per-model and per-domain overlap and ablation results

Neuron overlap and causal ablations both reveal modular organization: tasks within the same cognitive domain share more highly attributed neurons and depend more strongly on them than cross-domain tasks.

  • Per-model results: Within-domain neuron overlap exceeded cross-domain overlap in every model, with W/C ratios from 3.5× to 5.9× and p < 0.0001 in all six models.Within-domain overlap ranged from 11.1% to 14.2%, while cross-domain overlap ranged from 2.3% to 3.5%.
  • Per-model results: Causal ablation reduced accuracy by 18.2% to 26.4% for within-domain neurons versus 2.1% to 3.8% for matched cross-domain neurons.The resulting W/C ratios ranged from 6.1× to 12.3×, with a mean of 9.3 ± 2.7 and significance in every model.
  • Cross-model consistency: All six models, spanning four families and 24B to 123B parameters, independently showed modularity in both overlap and ablation analyses.The result therefore appears in individual models rather than only in a cross-model average.
  • Per-domain results: Across four domains and three analyses, the within-versus-cross asymmetry held in all cases and was significant in 10 of 12 comparisons.Social reasoning showed the same directional effect but fell just below significance in both directional ablations.
  • Per-domain results: Within-domain overlap exceeded cross-domain overlap in every cognitive domain, ranging from 8.6% to 14.2% within domains and 1% to 4.7% across domains.The within-versus-cross difference was significant in every domain.

5. Robustness to thresholding choices

The within-versus-cross-domain modularity pattern remains robust across attribution thresholds, although widening the threshold dilutes domain specificity. Ablation analyses show the same threshold-robust asymmetry in causal effects.

  • Neuron overlap: Four thresholds—top 0.05%, 0.1%, 1.0%, and 5.0%—were used to recompute overlap matrices and permutation tests across six models.The analyses used 10,000 permutations of task-to-domain labels.
  • Neuron overlap: The within-versus-cross-domain overlap difference remained highly significant at every threshold, with all permutation p-values below 0.0001.The 0.1% row reproduces the main-text values.
  • Neuron overlap: 5.4× at the top 0.05% threshold declined to 1.4× at the top 5.0% threshold as broader neuron sets diluted the within-domain signal.The reported 0.1% overlap values were 12.9% within-domain versus 3.0% cross-domain.
  • Ablation effects: At every threshold, within-domain neuron ablations caused substantially larger accuracy drops than cross-domain ablations, with all p-values below 0.0001.This test was conducted in Qwen2.5-32B-Instruct across both ablation directions.
  • Ablation effects: W/C ratios for ablation effects ranged from 2.6× to 12.9× across thresholds and directions, while widening thresholds increased drops in all conditions.The increase is expected because broader thresholds ablate more neurons.
  • Interpretation: The strongest domain-specific causal contributions are concentrated among the highest-attributed neurons, with broader sets adding progressively less domain-selective influence.This explains the declining within-versus-cross asymmetry as the threshold widens.

6. Robustness to ablation method choices

The modular ablation pattern is robust across intervention methods and depends on the model’s ability to perform the tasks. A weaker model shows only limited organization, concentrated in language.

  • Ablation methods: Mean, zero, and counterfactual ablation all produced larger accuracy drops when within-domain neurons were ablated than when cross-domain neurons were ablated.The pattern held across every domain and both ablation directions.
  • Ablation methods: Counterfactual ablation produced the largest accuracy drop because it replaces activations with values that actively mislead the model.Zero and mean ablation can instead push activations out of distribution.
  • Ablation methods: The modular pattern remained consistent across all three ablation methods and all four cognitive domains.This supports robustness to the choice of intervention.
  • Task performance: GPT-2 performed above chance on Language at 0.77 but below chance on Formal at 0.03, Physical at 0.05, and Social at 0.09.The both-correct metric has chance level 0.25.
  • Task performance: GPT-2’s within-domain effect was weaker than in the large models, with W/C = 2.67×, p = 0.0003, and Cohen’s d = 0.37 versus d ≥ 0.68 in all large models.Removing Language reduced GPT-2’s effect to W/C = 1.57×, p = 0.05, d = 0.17.
  • Task performance: The detectable GPT-2 organization was driven by Language, the only domain in which GPT-2 performed above chance.Removing other domains left smaller detectable effects, but all remained below the large-model floor.
  • Conclusion: The results indicate that modular organization is contingent on a model being able to perform the tasks, rather than being produced by the datasets or pipeline alone.The same data and pipeline recovered only weak organization in GPT-2.

8. Model architectures

The study analyzes six instruction-tuned transformer models ranging from approximately 24B to 123B parameters. Neurons are defined as units in transformer MLP intermediate layers.

  • Neuron definition: A model’s total analyzed neurons equal the number of transformer layers multiplied by the MLP intermediate dimension.Here, “neuron” refers to a unit in a transformer layer’s feed-forward MLP sublayer.
  • Model architectures: Mistral-Small-24B-Instruct-2501 has 40 transformer layers, 32 attention heads per layer, and an MLP intermediate dimension of 32,768.These specifications describe the smallest listed model.
  • Model architectures: OLMo-2-0325-32B-Instruct and Qwen2.5-32B-Instruct each have 64 transformer layers and an MLP intermediate dimension of 27,648.The listed models differ in other architectural specifications such as attention heads.
  • Model architectures: Llama-3.1-70B-Instruct and Qwen2.5-72B-Instruct each have 80 transformer layers and 64 attention heads per layer.Their listed MLP intermediate dimensions are 28,672 and 29,568, respectively.
  • Model architectures: Mistral-Large-Instruct-2407 has approximately 123 billion parameters, 88 transformer layers, and 96 attention heads per layer.Its MLP intermediate dimension is 28,672.

9. A qualitative inspection of modular ablation effects

Targeted ablations produce a qualitative double dissociation: removing language neurons disrupts form while preserving physical reasoning, whereas removing physical-reasoning neurons preserves form but disrupts physical conclusions. Additional analyses indicate that neuron modularity exceeds prompt semantic similarity.

  • Qualitative ablation: The qualitative test used no ablation, language-neuron ablation, and physical-reasoning-neuron ablation during free-form generation.A wider top-5% threshold was used so effects would be visible in a single generation.
  • Qualitative ablation: Ablating physical-reasoning neurons preserved grammatical well-formedness but produced incorrect physical conclusions on the same question.The two interventions therefore yielded opposite error profiles.
  • Qualitative ablation: In a rubber-band heat-aging question, ablating language neurons preserved physical reasoning but introduced syntactic and morphological errors.The ablated sets were the top 5% positively attributed neurons for each domain.
  • Semantic controls: The neuron-overlap matrix recovered the four cognitive domains with ARI = 0.78, exceeding SBERT 0.39, GloVe 0.36, TF-IDF 0.12, and Qwen input-token embeddings 0.04.The Qwen embedding baseline was not significant at p = 0.17.
  • Semantic controls: The domain effect on neuron overlap remained positive and significant after controlling for every semantic baseline tested.The reported coefficients were β = 0.38, 0.47, 0.14, and 0.42 for TF-IDF, GloVe, SBERT, and Qwen embeddings, respectively.
  • Semantic controls: These analyses support domain-specific computations beyond semantic similarity between task prompts.The semantic controls included lexical, word-embedding, sentence-embedding, and model input-token representations.

language processing (EMNLP-IJCNLP) (pp. 3982-3992).

This section lists prior work on automatic text retrieval, neural-network interpretation, linguistic evaluation, physical reasoning, theory of mind, reasoning environments, language models, and circuit-discovery methods.

  • Related work includes term-weighting approaches in automatic text retrieval and methods for analyzing neural networks for NLP.
  • Additional references address the temporo-parietal junction in theory of mind, verifiable-reward reasoning environments, and automated circuit discovery.
  • The cited literature covers physical reasoning in large language models, linguistic minimal-pair evaluation, and technical documentation for Qwen2.5.
Loading 2608.13567v1…