Stronger Alignment between Brain Activity and LLM Embeddings during Code Writing compared to Prose Writing

Zachary Karas; Catie Chang; Kevin Leach; Yu Huang

Source-linked AI summary

Stronger Alignment between Brain Activity and LLM Embeddings during Code Writing compared to Prose Writing

Zachary Karas, Catie Chang, Kevin Leach, Yu Huang

arXiv:2608.24900v1Jul 14, 2026cs.HC

TL;DR

Programming is societally important, but the cognitive processes supporting code writing and efforts to support developers remain insufficiently understood. Using voxelwise encoding models to compare LLM embedding alignment with brain activity during code and prose writing, the paper finds stronger alignment for code writing, especially in the right frontal pole.

Problem
Programming has major societal influence, while the cognitive processes supporting code writing remain incompletely understood and efforts to support developers face challenges.
Method
Voxelwise encoding models related LLM embeddings from 23 programmers’ keystrokes to fMRI-measured voxelwise BOLD activity during code and prose writing.
Results
LLM embeddings predicted brain activity better during code writing than prose writing, especially in the right frontal pole.
Takeaways & Limitations
The findings suggest stronger alignment between human and LLM representations during structured code generation, with implications for AI systems that predict code generation and support natural language tasks.
Takeaways & Limitations
Participants had to type without seeing the keyboard, which notably influenced some participants, and touch-typing ability was considered during recruitment.

Abstract

from arXiv · show

Programming is a critical skill underlying modern software systems, yet the cognitive processes supporting code writing are only beginning to be understood, limiting educational practices and developer tools. At the same time, Large Language Models (LLMs) are increasingly used to assist programming. These models themselves are not well understood and can exhibit undesirable behavior like introducing security vulnerabilities. Given evidence that some cognitive representations may be shared between LLMs and the brain, we seek to improve our understanding on both fronts by relating these two systems to one another. We used Voxelwise Encoding Models (VEMs) to relate LLM embeddings to brain activity measured with functional Magnetic Resonance Imaging (fMRI) during naturalistic writing tasks. Using participants' (n = 23) keystrokes as prompts, we extracted LLM embeddings to predict voxelwise Blood Oxygen Level Dependent (BOLD) signal, quantifying alignment as the correlation between predicted and recorded signal. To assess whether this alignment is specific to programming or generalizes to other generative processes, we compared code writing to prose writing. Alignment was strongest in the right frontal pole, and brain activity was significantly better predicted by LLM embeddings during code writing than prose writing (p < 0.001, FDR-corrected). Within participants, the best-modeled voxel locations for code writing were 66% consistent across LLM layers but varied substantially between participants (39% similarity). Our findings suggest stronger alignment between human and LLM representations during structured code generation, with implications for designing AI systems that predict code generation but support natural language tasks.

1 Introduction

Programming’s growing cognitive demands and the limitations of current AI tools motivate linking brain activity with LLM representations. This study introduces VEMs for naturalistic code and prose writing, finding stronger brain–LLM alignment during code writing and layer-consistent but participant-variable modeled regions.

Motivation: Growing software complexity increases the cognitive demands of writing, understanding, and debugging code, motivating better cognitive support for developers.Existing interventions include education promoting mistake prevention and decomposition of complex computational problems into steps.
Motivation: Current programming AI tools are poorly understood and can be inadequate, misaligned, or introduce security vulnerabilities and other undesirable behaviors.Reported problems also include changing tests to pass, prompt injection attacks, and unexpected behavior.
Research gap: Prior work links semantic representations in LLMs and the human brain, but no previous study had used VEMs to model brain activity during code writing.The study hypothesizes that code-writing processes rely on a continuous representational space that can be modeled with LLM embeddings.
Approach: n = 23 participants’ writing behavior was analyzed with VEMs to model voxelwise BOLD signal during naturalistic code and prose writing.The study introduces VEMs for programming tasks and systematically evaluates different study parameters across 55,200 VEMs.
Findings: p < 0.001, FDR-corrected: VEMs predicted voxelwise BOLD signal significantly better during code writing than prose writing.The right frontal pole was identified as a region whose activity could be grounded in participants’ writing behavior.
Findings: 66% consistency across model layers within participants contrasted with 39% similarity between participants for top-modeled code-writing regions.DeepSeek 6B and StarCoder 3B performed best for code writing, whereas StarCoder 7B performed best for prose writing.

2 Methods

The study used participants’ keystrokes as prompts for LLM embeddings and voxelwise encoding models to predict fMRI BOLD activity. The pipeline combined hemodynamic-delay modeling with standard fMRI preprocessing and aligned keystrokes for code–prose comparisons.

Voxelwise encoding models: LLM embeddings represented participants’ written stimuli and were combined and weighted to predict voxelwise BOLD signal at each timepoint.Unlike hand-designed stimulus features, the model used numerical vectors extracted from LLM representations.
Voxelwise encoding models: At each fMRI timepoint, participants’ keystrokes were provided as LLM prompts, and embeddings were extracted as representations of what they wrote.Delayed copies of feature vectors modeled the hemodynamic delay before ridge regression mapped them to voxel activity.
Dataset preparation: The analysis used previously recorded fMRI data from 23 participants who completed both code-writing and prose-writing tasks.The original sample contained 25 students, but two participants were excluded because they completed only one task.
Data preprocessing: fMRI preprocessing included equilibration-volume removal, motion correction, registration to MNI152, ICA-based artifact removal, nuisance regression, and temporal z-scoring.Low-quality physiological recordings limited direct physiological correction, so ICA was also used to address represented physiological noise.
Data preprocessing: Keystroke preprocessing aligned key presses with fMRI volumes, separated responses by question, and handled special characters such as arrows and backspaces.The resulting representations served as prompts for the LLMs.

3 Results

LLM embeddings predicted voxelwise brain activity significantly better during code writing than prose writing across parameter configurations. Top-modeled brain regions were more consistent across LLM layers within participants than between participants, with code-writing consistency of 66% across layers and 39% between participants.

Code Writing Versus Prose Writing: LLM embeddings predicted voxelwise BOLD signal significantly better during code writing than prose writing across a wide range of parameter combinations (p < 0.001, FDR-corrected).Correlations were higher for Code than Prose for every tested parameter configuration.
Within- Versus Between-Participant Similarity: Within-participant similarity exceeded between-participant similarity across LLMs (µ = 0.543, σ = 0.108 versus µ = 0.397, σ = 0.017; t = 3.188, p < 0.05, d = 1.301).The authors report that this pattern generalized across most state-of-the-art LLMs considered.
Consistency Across LLM Layers: 66% of top-modeled brain regions were shared across LLM layers for code writing within participants, compared with 62% for prose writing.These results indicate substantial within-participant consistency across layers.
Between-Participant Similarity: 39% of top-modeled voxel locations were shared between participants for code writing, compared with 41% for prose writing.Average Jaccard similarity was 0.388 (σ = 0.149) for Code and 0.414 (σ = 0.137) for Prose.

4 Discussion

The discussion argues that LLM embeddings model brain activity more feasibly during code than prose writing, with the right frontal pole showing consistent alignment and top-modeled regions consistent across layers but distinctive between participants. It also outlines implications for AI tools and limitations involving missing keystroke periods, editing measures, language scope, and touch-typing demands.

Code versus Prose: LLM-embedding models were more feasible for code writing than prose, potentially because code is more regular and structured.The authors suggest code may occupy a more sparse continuous representational space than natural language.
Right Frontal Pole: The right frontal pole consistently aligned well with keystroke-derived LLM embeddings, especially among participants who performed well and were well modeled.The discussion links this region to high-order tasks and suggests its activity can be grounded in participants’ writing behavior.
Layer Consistency: Top-modeled brain regions were consistent across LLM layers within participants but distinctive between participants.The authors suggest layer coordination may support cohesive responses and that modern LLM representations may be distributed across layers.
Interpretation: Modeling alignment may indicate representational similarity rather than task importance, because regions important to a task may not align closely with an LLM.The discussion notes that alignment depends on whether a region’s cognitive representation is inherently close to the LLM’s representation.
Implications: Closer brain–LLM alignment for code could support tools that predict code generation, while prose tools could assist with organizing thoughts, vocabulary, and structure.Proposed applications include programming education and software engineering tools that detect deviations from expected or project-specific patterns.
Limitations: The study’s limitations include static VEM estimates during non-typing periods, rudimentary editing measures, single programming and natural languages, and scanner-based touch-typing demands.The authors recommend richer information sources, improved editing measures, additional languages, and considering touch-typing ability during recruitment.

5 Conclusions

LLM embeddings of participants’ keystrokes predicted brain activity more accurately during code writing than prose writing, especially in the right frontal pole. Within-participant consistency across LLM layers contrasted with substantial between-participant variability, supporting embeddings as a tool for studying cognition in writing.

5 Conclusions: LLM embeddings of participants’ keystrokes predicted brain activity better for code writing than prose writing, especially in the right frontal pole.The right frontal pole is implicated in high-level goal planning, prospective memory, and information integration.
5 Conclusions: LLM embeddings of participants’ keystrokes are a viable tool for studying cognition associated with code and prose writing.The findings have implications for AI tools that more reliably predict code completion than support natural language tasks.
5 Conclusions: 66% of brain regions best modeled by LLM layers were the same within participants for code writing, compared with 62% for prose writing.This within-participant consistency suggests information may be represented across layers within modern LLMs.
5 Conclusions: 39% of brain regions best modeled by LLM embeddings were the same between participants for code writing, compared with 41% for prose writing.The variability across participants suggests each individual may have a unique fingerprint in how brain activity can be modeled using LLM embeddings.

Data and Code Availability

Neuroimaging data are openly available upon request, and the analysis code is available at the listed GitHub repository.

Neuroimaging data used in the study is openly available upon request.
Analysis code can be found at https://github.com/largehappygroup/fmri model.Inquiries are welcome and can be directed to the study authors.

Supplementary Material

Supplementary analyses normalized top-10K voxel counts by region size to identify specialized brain regions and examined participant- and layer-level consistency. Normalization increased variability, while code showed higher between-participant similarity than prose and low-performing groups were primarily modeled in occipital regions.

Participant groups: Low-performing and poorly modeled participants had top regions primarily clustered in occipital areas, whereas high-performing and well-modeled participants showed parietal, occipital, and frontal clustering.The results suggest embeddings more reliably predict low-level sensory activity than higher-order planning or spatial-temporal processes for the lower-performing groups.
Layer consistency: 40%: normalized top-modeled regions were approximately layer-consistent, with average Jaccard indices of 0.405 (σ = 0.314) for Code and 0.440 (σ = 0.296) for Prose.Code–Prose differences were not significant (t = 0.381, p = 0.707), and Jaccard indices did not significantly correlate with modeling performance for Code (r = −0.033, p = 0.883) or Prose (r = 0.090, p = 0.681).