PaperlayerView exact revision on arXiv

Source-linked AI summary

Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models

Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, Ji-Rong Wen

arXiv:2402.16438v2cs.CL

TL;DR

The paper addresses limited understanding of how LLMs process diverse languages internally. It uses LAPE to identify language-specific neurons and finds that a small subset, mainly in top and bottom layers, influences language proficiency. It also demonstrates the potential to steer output language by selectively activating or deactivating these neurons.

  • Problem

    How LLMs process diverse languages at the model-composition level remains insufficiently understood, including the role of language-specific regions.

    Source: §1 · p. 1

  • Method

    LAPE assesses individual neurons’ activation responses across languages and selects neurons with a propensity to activate for one or two languages.

    Source: §5 · p. 9

  • Results

    A small subset of neurons, mainly in top and bottom layers, significantly influences LLM proficiency in processing particular languages.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Selective activation or deactivation of language-specific neurons shows potential for steering output language and mitigating off-target replies.

    Source: §1 · p. 2

    LAPE depends on multiple languages being present; with only one language, an absolute threshold for neuron language-relatedness is not feasible.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language models (LLMs) demonstrate remarkable multilingual capabilities without being pre-trained on specially curated multilingual parallel corpora. It remains a challenging problem to explain the underlying mechanisms by which LLMs process multilingual texts. In this paper, we delve into the composition of Transformer architectures in LLMs to pinpoint language-specific regions. Specially, we propose a novel detection method, language activation probability entropy (LAPE), to identify language-specific neurons within LLMs. Based on LAPE, we conduct comprehensive experiments on several representative LLMs, such as LLaMA-2, BLOOM, and Mistral. Our findings indicate that LLMs' proficiency in processing a particular language is predominantly due to a small subset of neurons, primarily situated in the models' top and bottom layers. Furthermore, we showcase the feasibility to "steer" the output language of LLMs by selectively activating or deactivating language-specific neurons. Our research provides important evidence to the understanding and exploration of the multilingual capabilities of LLMs.

1 Introduction

The introduction frames language-specific regions as an underexplored part of LLM multilingual processing and proposes LAPE to identify language-specific neurons. It reports that a small, top- and bottom-layer subset affects language proficiency and may be manipulated to steer output language.

1 Introduction

  • Multilingual capabilities matter for information access across linguistic divides, but composition-level mechanisms and language-specific LLM regions remain underexplored.Earlier work examined cross-lingual transfer factors such as overlapping tokens and shared parameters.Source: §1 · pp. 1–2
  • LAPE identifies language-specific neurons by selecting neurons with low language activation probability entropy, and the study examines how these regions relate to multilingual capabilities.The authors describe this as the first study of language-specific regions inside LLMs and make the identified neurons and code available.Source: §1 · p. 2
  • Deactivating a minuscule proportion of language-specific neurons can markedly degrade an LLM’s understanding and generation in that language.The authors report that language proficiency can be significantly impacted by this small subset.Source: §1 · p. 2
  • Language-specific neurons are concentrated mainly in the bottom and top layers, which are associated with input processing and language-specific token projection, respectively.The bottom layers map diverse-language inputs toward a unified semantic space, while the top layers project processed semantics into each language’s vocabulary.Source: §1 · p. 2
  • Selective activation or deactivation of language-specific neurons may steer output language and help mitigate off-target English replies to Chinese queries.The authors also suggest this may stimulate cross-lingual generation.Source: §1 · p. 2

2 Identifying Language-Specific Regions

The section presents FFN neurons as candidates for multilingual processing and introduces LAPE to identify neurons whose activation probabilities favor particular languages. It describes how language-specific neurons are operationally selected.

2.1 Background

  • Transformer layers contain multi-head self-attention and FFN modules; each FFN has 4d neurons, defined through a W1-column linear transformation followed by a nonlinear activation.Source: §2.1 · p. 2

2.2 Language Activation Probability Entropy

  • Prior findings that FFN neurons can store factual knowledge, encode positional information, or respond to syntactic triggers motivate the hypothesis that some specialize in multilingual processing.The cited examples include Dai et al. (2022), Voita et al. (2023), and Gurnee et al. (2024).Source: §2.2 · p. 3
  • The method estimates each neuron's activation probability from language-specific text using activations above zero, then L1-normalizes the per-language probabilities before computing entropy.Source: §2.2 · p. 3
  • Low-LAPE neurons are designated language-specific because they favor activation for one or two languages while activating less for others.LAPE quantifies each neuron's language activation reaction using entropy.Source: §2.2 · p. 3
  • Using Wikipedia corpora with 100 million tokens per language, the procedure selects neurons in the lowest 1% of LAPE scores and applies an activation-probability threshold to exclude negligible activations.Source: §2.2 · p. 3

3 Experiments

The experiments test LAPE against alternative neuron-identification methods across multilingual language modeling and generation. They find that language-specific neurons are concentrated at the bottom and top layers, and that manipulating them can affect or steer output language.

3.1 Experimental Setup

  • Experiments compare LAPE with probability-, activation-value-, parameter-variation, and random-selection methods, using neuron perturbations to assess multilingual modeling and generation.The study includes LLaMA-2, BLOOM, and other model types, with Wikipedia perplexity and open-ended generation evaluations.Source: §3.1 · pp. 3–4, §3.2 · p. 4

3.2 Main Perturbation Experiments

  • LAPE’s perturbations chiefly raise PPL for the corresponding language, while LAVE shows cross-language interference and other methods lack clear language-specific patterns.The comparison measures PPL changes across languages after perturbing identified regions.Source: Figure 2 · p. 4, §3.2 · pp. 4–5
  • Language-specific degradation patterns appear across LLaMA-2, BLOOM, OPT, Mistral, and Phi-2; larger models tend to have more single-language-specialized neurons.Chinese and Japanese neurons overlap by approximately 25%, which the authors suggest may relate to their shared characters.Source: Figure 3 · p. 5, §3.2 · p. 5

3.3.1 Distribution and Identification Ratio

  • About 23K language-specific neurons are identified in LLaMA-2 70B, with fewer English-specific neurons than for other languages.Per-language counts can sum to more than 23K because neurons may be shared; the authors speculate English’s smaller count reflects its dominance in the model.Source: §3.3.1 · p. 6, Table 3 · p. 6
  • Deactivating French-specific neurons significantly raises French PPL, while effects on other languages are limited except for Spanish.The analysis varies the proportion of selected neurons from 0 to 10%.Source: Figure 4 · p. 6, §3.3.1 · p. 6

3.3.2 Structural Distribution Analysis

  • About 7,000 language-specific neurons occur in LLaMA-2 70B’s second layer, versus about 100 in layers 5–47 and over 1,000 in each final four layers.The authors relate the U-shaped distribution to cross-language sentence-embedding similarity: bottom-layer semantic alignment and top-layer vocabulary mapping require language-specific neurons.Source: §3.3.2 · pp. 6–7, Figure 5 · p. 7, Figure 6 · p. 7

3.3.3 Language Dominance Analysis

  • English has the highest mean SES as the target language in LLaMA-2 70B, whereas English and Chinese both show dominance in multilingual BLOOM 170B.The authors describe low-resource languages as centered around high-resource languages.Source: Figure 7 · p. 7, §3.3.3 · p. 7

3.3.4 Case Study

  • Activating language-specific neurons increases expected-language responses and GPT-4-rated generation quality, while switching from Spanish to Chinese neurons enables fluent Chinese answers to Spanish questions.The case studies examine both steering into the prompted language and cross-lingual generation.Source: §3.3.4 · p. 8, Table 4 · p. 8

4 Related Work

Prior work examines factors shaping multilingual ability and shared model components, while language-specific components in LLMs remain under-investigated. The paper situates its method among neuron analyses of specialized contexts and linguistic properties.

4 Related Work

5 Conclusion

The paper introduces LAPE to identify language-specific neurons and reports that a small subset, concentrated mainly in top and bottom layers, significantly influences language proficiency. It also demonstrates output-language steering through selective neuron activation or deactivation.

5 Conclusion

  • LAPE identifies language-specific neurons by selecting those with activation propensity for one or two languages.Source: §5 · p. 9
  • A small subset of neurons significantly influences an LLM’s proficiency in processing different languages, and these neurons are mainly in top and bottom layers.Source: §5 · p. 9
  • Selective enabling or disabling of language-specific neurons can direct an LLM’s output language.Source: §5 · p. 9
  • Future work could use these findings to support knowledge transfer between major and minor languages and continual pre-training for specific languages.Source: §5 · p. 9

Limitations

The authors identify limits in LAPE’s language-relative setting and leave language-resource distinctions, large language sets, and proficiency enhancement for further investigation.

Limitations

  • LAPE-based identification depends on multiple languages; with only one language, an absolute neuron language-relatedness threshold is not feasible.Source: Limitations · p. 9
  • Criteria for distinguishing high-resource from low-resource languages within a model warrant further investigation.Source: Limitations · p. 9
  • Handling many languages and their differences, and using identified neurons to enhance multilingual proficiency, remain avenues for future research.Source: Limitations · p. 9

A Appendix

The appendix lists pre-training corpus language statistics and per-layer language-specific neuron counts for several models, and includes results from random neuron deactivation.

A Appendix

Loading 2402.16438v2…