PaperlayerView exact revision on arXiv

Source-linked AI summary

Distribution-aware Language Neuron Identification in Multilingual Large Language Models

Minjun Kim, Inho Won, Junghun Yuk, Dongyeon Kim, Jihyo Kim, KyungTae Lim

arXiv:2609.10993v1cs.CL

TL;DR

Existing language-neuron identifiers reduce continuous activations to positive-versus-inactive statistics, which can miss negative-regime information and relationships among language distributions. DLN instead clusters languages using pairwise overlap coefficients over full activation distributions. Across two models and two held-out corpora, it more effectively isolates language-specific causal effects, achieving up to 4.9× higher on-target damage per neuron while preserving off-target performance.

  • Problem

    Binary activation statistics can discard negative activations and the shape and relational structure of multilingual activation distributions.

  • Method

    DLN selects language neurons by clustering languages according to pairwise overlap coefficients between their full per-language activation distributions.

  • Results

    Up to 4.9× higher on-target language damage per neuron is achieved while preserving off-target performance, with effectiveness across SLNs and MLNs.

  • Takeaways & Limitations

    DLN identifies language-specific causal effects and shared language coalitions while capturing neurons operating in negative activation regimes and beyond the standard gate-activation site.

  • Takeaways & Limitations

    The study focuses exclusively on autoregressive Transformer models with GLU feed-forward blocks using SiLU, leaving generalization to standard MLP architectures unexplored.

Abstract

from arXiv · show

Multilingual large language models (mLLMs) contain a small fraction of feed-forward neurons that are sensitive to particular languages, commonly termed language-specific neurons. Existing work measures language specificity using the entropy of each neuron's language-wise probabilities of being active, where a neuron is considered active when its activation value is positive. However, this approach may not fully capture the multilingual nature of mLLMs, where language representations are distributional and mutually related. We propose Distribution-aware Language Neuron selection, which leverages pairwise relationships between per-language activation distributions over the full activation range, including negative values. Specifically, we quantify each neuron's language specificity by clustering languages using pairwise overlap coefficients between their activation distributions. Across two mLLMs and two held-out corpora, our identifier more effectively isolates language-specific causal effects, yielding up to 4.9$\times$ higher on-target language damage per neuron while preserving off-target language performance.

1 Introduction

mLLMs contain language-sensitive neurons, but binary positive-activation statistics can miss distributional distinctions and relationships among languages. DLN selection addresses this by clustering full activation distributions, improving causal isolation for single- and multi-language neurons.

  • 1 Introduction: Binary activation probabilities discard negative-activation information and fail to preserve the shapes and relational structure of per-language distributions.These limitations motivate a distribution-aware alternative for multilingual representations.
  • 1 Introduction: DLN selection identifies language neurons by clustering pairwise relationships among full per-language activation distributions, including negative values.It reframes language neurons as neurons whose activation distributions differ across languages.
  • 1 Introduction: DLN identifies both single-language neurons and structurally coupled multi-language coalitions, including shared language groups.The framework distinguishes neurons associated with one language from those associated with multiple languages.
  • 1 Introduction: Up to 4.9× higher on-target language damage per neuron is achieved by ablating DLN-selected SLNs while preserving off-target performance.Evaluations cover Llama-3.1-8B and SmolLM3-3B-Base on two held-out corpora.
  • 1 Introduction: Full-distribution analysis reveals negative-regime neurons and supports interventions beyond the standard gate-activation site.The reported analysis identifies negative-regime neurons and maintains selectivity at the gate-up product site.

2 Related Work

Prior multilingual neuron studies extend neuron-level interpretability by identifying language-specific units with binary activation statistics. Related work motivates examining richer activation distributions because similar activation rates can conceal distinct language structures.

  • 2 Related Work: Multilingual models share parameters across languages while preserving language identity, motivating questions about language-specific and language-shared computation.Prior findings also describe partially overlapping representational subspaces and possible English-centric pivot representations.
  • 2 Related Work: Neuron-level interpretability has shown that individual Transformer neurons can encode factual, syntactic, semantic, and task-relevant information.Multilingual language-neuron identification extends this line of analysis to language-specific behavior.
  • 2 Related Work: Prior work identifies language-specific neurons using per-language activation probabilities and entropy based on whether activation values are positive.These methods classify neurons according to how selectively they activate across languages.
  • 2 Related Work: Binary activation statistics can miss distinctions because languages with similar activation probabilities may still have different activation-value distributions.The proposed contrast is analysis of full per-language activation distributions.

3 Distribution-aware Language Neurons

DLN identifies language neurons from continuous per-language activation distributions by comparing their pairwise overlap and selecting a low-overlap language cluster. The method distinguishes single- and multi-language neurons while retaining distributional structure discarded by binary statistics.

  • 3 Distribution-aware Language Neurons: Full-range distributions preserve negative activation information and expose separable activation regimes that entropy-based binary methods can miss.The framework therefore supports distributional identification of both single-language and multi-language neurons.
  • 3.1 Distribution Overlap Coefficient Matrix: DLN computes pairwise overlap coefficients between continuous per-language activation densities to quantify how similarly each neuron behaves across languages.The overlap matrix summarizes shared areas between language-specific activation distributions.
  • 3.2 Cluster-based Neuron Classification: The method partitions languages into two clusters by minimizing the maximum between-cluster overlap, equivalent to single-linkage clustering with two clusters.This is a selection decision rather than an assumption that distributions are bimodal.
  • 3.2 Cluster-based Neuron Classification: The smaller cluster becomes the candidate language set, while the larger cluster serves as a reference group with relatively similar activation behavior.The method assumes that languages with distinct activation patterns are most strongly associated with the neuron.
  • 3.3 Neuron Selection Criterion: Selected neurons are categorized as SLNs when the candidate set contains one language and MLNs when it contains multiple languages.Selection requires all between-cluster overlap coefficients to remain below threshold τ.
  • 3.3 Neuron Selection Criterion: The overlap threshold τ is calibrated as a percentile of overlap coefficients computed across all neurons because overlap values vary across models.This makes the threshold relative to the model’s empirical overlap distribution.

4 Experiments

Experiments compare distribution-aware SLN and MLN selections with entropy-based baselines using mean-patch interventions across models, corpora, languages, and downstream tasks. The proposed selections concentrate causal damage on target languages while preserving off-target performance and reveal shared multi-language neuron structure.

  • Single-language neurons: 4.9× and 5.4× larger on-target ΔNLL per neuron than LAPE on WIKI and FLORES, respectively, with SLN also exceeding LSN.Averaged over seven languages on Llama-3.1-8B, SLN’s per-neuron damage is 3.1× and 3.2× larger than LSN on WIKI and FLORES.
  • Multi-language neurons: 6.85× and 7.12× greater within-cluster ΔNLL per neuron than LRN on WIKI and FLORES, while MLN uses 465 neurons versus LRN’s 1,329.The comparison averages on-target damage across the four most-populated multi-language clusters.
  • Multi-language neurons: Chinese language neurons are largely assigned to the {zh, ja} MLN cluster, recovering Chinese damage that is weak under all three single-language identifiers.The {zh, ja} cluster yields Chinese ΔNLL of 1.24/1.12 on WIKI/FLORES and within-cluster Japanese damage of 0.92/0.89.
  • Selectivity robustness: Selectivity exceeds LAPE and LSN at every percentile threshold across both models and corpora, with the LAPE gap reaching 20–32× on WIKI and 18–26× on FLORES for SmolLM3-3B-Base.On Llama-3.1-8B, the gap over LAPE is 4–8×, while the gap over LSN is 1.1–1.9×; selectivity decreases monotonically as P increases.
  • Downstream tasks: On Belebele and MGSM, SLNs produce the largest on-target accuracy drop for every model-task combination while off-target accuracy remains essentially unchanged.On Belebele, the intervention removes 22%/27% of the above-chance target-language margin for Llama-3.1-8B/SmolLM3-3B-Base, with no off-target loss exceeding 2.40 points.

5 Analysis

The analysis shows that distribution-aware neuron selection identifies diverse and functionally distinct language neurons, including negative-region neurons and neurons operating at the causally operative z-site. These selections produce stronger language-selective effects than activation-probability baselines while preserving off-target behavior.

  • Comparison between LAPE and SLN: 2.86× and 4.71× higher on-target ΔNLL than LAPE \ SLN are achieved by SLN \ LAPE on Llama-3.1-8B and SmolLM3-3B-Base, respectively.The intersection of LAPE and SLN has the highest selectivity, followed by SLN \ LAPE and LAPE \ SLN.
  • Distributional diversity of SLNs: Distribution-aware selection identifies four SLN patterns, including target-negative, all-positive magnitude-shift, and all-negative distributionally separated neurons beyond indicator-based methods.The four patterns are only target positive, only target negative, all positive, and all negative.
  • Functional role of negative-region neurons: Negative-region neurons selectively support switching away from English: they are effective for non-English targets, whereas positive-region neurons and LAPE support English outputs.For English outputs, N + {en} and LAPE achieve accuracies of 0.96 and 0.97, while the negative subset reaches 0.71; the pattern reverses for non-English targets.
  • Functional role of negative-region neurons: Matched-displacement controls produce only about 1% of the SLN effect, indicating that large on-target ΔNLL is not explained by intervention displacement alone.Randomly sampled controls were count-matched by layer and shifted by the same 1.6–2.2σ displacement as matched SLNs.
  • Extension to the z-site: At the z-site, the method retains substantial advantage despite a 29–37% efficacy decrease, outperforming LAPE by 25–44× and LSN by 15–21×.The cross-language mean spread is about 4× narrower at z than at a, producing a uniform intervention-side reduction while preserving the method ordering.

6 Conclusion

The paper proposes distribution-aware language-neuron selection to model full activation distributions and their pairwise relationships. It identifies influential single- and multi-language neurons, reveals functional roles for negative activations, and extends to the z-site.

  • Distribution-aware selection models full activation ranges and pairwise relationships among language distributions to identify language neurons in mLLMs.
  • The method identifies multi-language neurons that form coherent language groups with relatively large performance degradation under intervention.
  • Negative-regime language neurons play functional roles in language switching, and the method also applies to the gate-up product z-site.

Limitations

The study’s evidence is bounded by its language coverage, GLU architecture, and reduced intervention magnitude at the z-site.

  • Experiments sample only seven languages, a tiny fraction of the languages represented in modern multilingual LLM training.
  • The method is evaluated only on autoregressive Transformer models with GLU feed-forward blocks and SiLU, leaving standard MLP generalization unexplored.
  • At the z-site, a cross-language mean spread approximately four times smaller than at the a-site causes a uniform drop in mean-patching effectiveness.

Use of AI Assistants

The paper reports AI-assisted language polishing and documents the authors’ review of those suggestions.

  • Gemini was used for grammar correction and sentence-level wording suggestions, with all suggestions reviewed and revised by the authors.

B Comparison under Different Interventions

Under zero-out intervention, DLN-selected neurons generally outperform LAPE despite using fewer neurons, while mean-patching reveals distributional language-specificity that zeroing can miss.

  • Comparison under Different Interventions: DLN identifies more effective language neurons than LAPE under zero-out intervention in most cases, despite selecting fewer neurons.The comparison uses zero-out intervention and evaluates on-target ΔNLL.
  • Comparison under Different Interventions: English SLNs produce much larger ΔNLL increases with mean-patching than zero-out, whereas LAPE changes little across interventions.This contrast indicates that the intervention choice matters differently for the two identifiers.
  • Comparison under Different Interventions: 63–72% of English SLNs have negative target-language means, so zeroing can remain near the natural target manifold while mean-patching moves activations toward off-target values.English target means lie within approximately 0.5σ of zero, while the off-target mean is approximately 1.5σ away.
  • Comparison under Different Interventions: The zero-ablation results confirm that the distributional identifier’s full advantage appears under its matched intervention, because rate-based interventions hide distribution-encoded specificity.The passage links the result to coexistence of activation-rate and distributional forms of language-specificity.

C Site-comparison: Supplementary Results

Supplementary site comparisons show that distribution-aware selection remains selective when moved beyond the standard gate-activation site, whereas indicator baselines lose most of their selectivity.

  • Site-comparison: Supplementary Results: At the z-site, every layer has activation-positive rate within 0.002 of 0.5, so sign statistics provide no cross-layer or cross-language signal there.At the a-site, rates range from 0.16–0.48 on Llama-3.1-8B and 0.18–0.46 on SmolLM3-3B-Base.
  • Site-comparison: Supplementary Results: At the controlled (z, a) configuration, DLN is 25–55× stronger than LAPE and 15–29× stronger than LSN across models and held-out corpora.This comparison varies the selection site while keeping the patch site fixed.
  • Site-comparison: Supplementary Results: Moving selection from a to z makes LAPE lose 74–86% and LSN lose 89–94% of selectivity, while DLN retains 50–71%.The full grid covers both held-out corpora and all three selection/patch-site configurations.
  • Site-comparison: Supplementary Results: The LAPE∩SLN subset has concentrated diagonal effects, while SLN\LAPE often exceeds LAPE\SLN on-target damage, especially for English.LAPE\SLN is larger but has weaker, more diffuse diagonal and off-diagonal effects.

D.2 Impact of MLNs on Language-specific Performance

MLN clusters produce language-linked damage across models, with stronger effects when clusters account for more language-associated neurons; the pattern persists at larger scale and in Aya-23-8B.

  • D.2 Impact of MLNs on Language-specific Performance: MLN-cluster ablations generally degrade the performance of their included languages across Llama-3.1-8B, SmolLM3-3B-Base, and Qwen3-4B-Base.The comparison uses the four most frequent MLN clusters on WIKI and FLORES.
  • D.2 Impact of MLNs on Language-specific Performance: Clusters covering a larger proportion of a language’s associated neurons tend to cause larger performance drops, exemplified by {zh, ja} versus {fr, es} and {id, vi}.SmolLM3-3B-Base is an exception for {id, vi}, whose languages are unsupported by that model.
  • D.2 Impact of MLNs on Language-specific Performance: On Llama-3.1-70B, {zh, ja} again shows substantial degradation while {fr, es} shows relatively minor degradation, preserving the smaller-model trend.The authors report that the MLN dynamics are not constrained by model scale.
  • D.2 Impact of MLNs on Language-specific Performance: On Aya-23-8B, DLN yields the largest on-target damage for all seven languages and both corpora while maintaining the lowest mean off-target damage.Mean selectivity is 254 versus 133 for LSN and 24 for LAPE on WIKI, and 691 versus 350 and 38 on FLORES.
  • D.2 Impact of MLNs on Language-specific Performance: For English-to-other-language transfer, LAPE approaches zero while DLN reaches accuracies from 0.599 to 0.952 depending on architecture.The passage attributes this pattern primarily to negative activation regions.
  • D.2 Impact of MLNs on Language-specific Performance: DLN remains 9–14× more selective than LAPE throughout the extended threshold sweep, while SLN on-target damage rises from 1.47 to 1.64.Within the SLN stratum, off-target damage stays below 0.02 nats as the percentile increases.

E Robustness of the Identifier Design

The identifier is robust to histogram resolution, bounded distance choices, and linkage rules, but scale-sensitive metrics select larger, less effective pools; the fixed two-cluster design remains a limitation.

  • E Robustness of the Identifier Design: Jensen–Shannon, squared Hellinger, and Kolmogorov–Smirnov produce Jaccard overlap of 0.92–0.99 and selectivity within approximately 10% of the overlap statistic.They also preserve the multi-language structure, while linkage choices yield Jaccard overlap of 0.96–0.99.
  • E Robustness of the Identifier Design: Wasserstein-1 and energy distance inflate pools by 1.6–1.9×, roughly halve selectivity, and distort MLN structure because their rankings favor high-variance neurons.Truncating these pools to the overlap method’s size makes performance worse rather than better.
  • E Robustness of the Identifier Design: The overlap coefficient equals one minus total variation distance, making the threshold interpretable as shared probability mass and enabling computation over 458k neurons in 10 seconds on CPU.Jensen–Shannon takes 96 seconds under the reported comparison.
  • E Robustness of the Identifier Design: Only 1.4% of selected neurons admit a threshold-clearing residual split, while no rejected neurons admit a three-group structure under the selection criterion.Choosing the number of clusters by within-cluster cohesion is left for future work.

G.2 Per-language Downstream Results

Per-language downstream results show that DLN-selected SLNs generally produce larger target-language accuracy drops than baselines while retaining limited off-target effects, with Chinese as the main exception because its signal is concentrated in MLNs.

  • Per-language accuracy results: 6.56 and 12.00 points on Llama-3.1-8B and 3.22 and 10.00 on SmolLM3-3B-Base are removed by English SLNs, versus at most 0.33 and 3.60 for baselines.These results cover Belebele and MGSM, respectively, and reproduce the English gap observed in ΔNLL.
  • Per-language accuracy results: Chinese is the one target where DLN SLNs trail LAPE, at 0.11 versus 3.33 on Belebele and −0.40 versus 8.00 on MGSM for Llama-3.1-8B.Most Chinese language neurons belong to the {zh, ja} MLN cluster, so the Chinese SLN pool omits the main causal pool.
  • Per-language accuracy results: Outside Chinese, baselines exceed DLN's raw on-target drop in only 2 of 20 remaining cells, while DLN still achieves 1.06× and 2.27× larger per-neuron effects there.The exceptions are Indonesian on Belebele and Japanese on MGSM with Llama-3.1-8B.
  • Statistical checks: DLN SLNs have the smallest McNemar-test p-values in all four settings, reaching p ≤4×10^-25 on Belebele and p ≤1×10^-9 on MGSM.Ten of twelve off-target entries are not significant at the 0.05 level, although the pooled off-target test is anti-conservative because evaluation items are reused.
  • Activation-distribution examples: The four activation-pattern examples show that DLN identifies SLNs missed by LAPE when target and non-target languages differ by sign or magnitude, including entirely negative distributions.The canonical positive-target/negative-rest pattern is identified by both methods, whereas cases 2–4 are DLN-only.
  • Language-population structure: Across three models, SLN and MLN distributions are similar, with Indonesian having the highest SLN proportion and Chinese the lowest, while frequent MLN clusters recur across models.The {en, fr, es} cluster is most frequent for English in two models, whereas Qwen3-4B-Base most frequently uses {en, zh}.
Loading 2609.10993v1…