PaperlayerView exact revision on arXiv

Source-linked AI summary

How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders

Tatsuro Inaba, Go Kamoda, Kentaro Inui, Masaru Isonuma, Yusuke Miyao, Yohei Oseki, Benjamin Heinzerling, Yu Takagi

arXiv:2503.06394v2cs.CLcs.LG

TL;DR

The study asks how bilingual representations emerge during pretraining and uses sparse autoencoders to trace them across stages, layers, and model sizes. It finds that alignment develops over training, especially in the mid-layers of larger models, and that injected bilingual representations improve a mid-training model’s performance.

  • Problem

    How bilingual capabilities emerge during pretraining, including whether languages are learned separately before alignment and how alignment varies across layers and model sizes, remains unclear.

    Source: §1 · p. 1

  • Method

    The study uses sparse autoencoders to trace language-specific and bilingual features, then selectively injects decomposed representations from a fully trained model into a mid-training model.

    Source: §1 · p. 2, §5.1 · p. 7

  • Results

    English-Japanese models first learn languages separately, then gradually align, with stronger mid-layer alignment in larger models; bilingual representations improve both languages’ mid-training performance.

    Source: §1 · p. 1, §5.3 · p. 7

  • Takeaways & Limitations

    Sparse autoencoders can serve as tools for both analyzing and manipulating language-model representations.

    Source: §7 · p. 8

    The findings may not generalize beyond English-Japanese, and SAE interpretability concerns and imperfect reconstruction mean analyses approximate model representations.

    Source: §8 · p. 8

Abstract

from arXiv · show

This study explores how bilingual language models develop complex internal representations. We employ sparse autoencoders to analyze internal representations of bilingual language models with a focus on the effects of training steps, layers, and model sizes. Our analysis shows that language models first learn languages separately, and then gradually form bilingual alignments, particularly in the mid layers. We also found that this bilingual tendency is stronger in larger models. Building on these findings, we demonstrate the critical role of bilingual representations in model performance by employing a novel method that integrates decomposed representations from a fully trained model into a mid-training model. Our results provide insights into how language models acquire bilingual capabilities.

1 Introduction

The study traces how English-Japanese bilingual representations emerge during pretraining, using sparse autoencoders to examine training stages, layers, and model sizes. It reports that bilingual alignment grows over training and that injected bilingual representations enhance a mid-training model’s performance.

1 Introduction

  • SAEs are used to track language-specific, bilingual, and semantic features across training stages and layers in decoder-only English-Japanese models of varied sizes.The approach aims to capture fine-grained distinctions among these feature types.Source: §1 · p. 1
  • English-Japanese language models initially learn languages independently, then gradually develop bilingual alignment.Source: §1 · p. 1
  • Bilingual alignments are more prominently captured in model mid-layers.Source: §1 · p. 1
  • Larger models exhibit stronger bilingual alignment than smaller models.Source: §1 · p. 1
  • Injecting bilingual representations from a fully trained model enhances a mid-training model’s performance.The method decomposes representations into English-specific, Japanese-specific, and bilingual types before selectively injecting them.Source: §1 · p. 2

2 Sparse Autoencoders

The section introduces sparse autoencoders and the TopK-SAE variant used to represent inputs through sparse hidden features. It describes the encoder and decoder, their initialization, and how TopK selects features for reconstruction.

2 Sparse Autoencoders

  • A sparse autoencoder enforces a sparsity constraint on its hidden layer, while TopK-SAE applies a TopK activation function there.TopK-SAE is described as easier to train than a ReLU-based SAE while maintaining sparsity and achieving higher reconstruction performance.Source: §2 · p. 2
  • The encoder and decoder are learned linear layers, with the decoder initialized from the encoder transpose and the bias from the input data’s geometric median.Source: §2 · p. 2
  • The training objective is mean squared error, and TopK-SAE is controlled by hidden dimension n and the number K of active dimensions.Source: §2 · p. 2
  • TopK-SAE selects K of n decoder vectors and uses their weighted sum to reconstruct the input; selected encoder dimensions are called activated features.Source: §2 · p. 2

3 Experiments

The experiments analyze English–Japanese model representations with SAEs, measuring feature activation patterns and language and semantic selectivity. The setup and metrics are designed to trace how individual features behave across languages and meanings.

3.1 Experimental Setup

  • The analysis covers five LLM-jp model sizes, while SAEs are trained on balanced English and Japanese Wikipedia subsets and evaluated on separate balanced subsets.The models were trained on the 1.7T-token LLM-jp Corpus v32; SAE training uses 100M tokens and evaluation uses 10M tokens.Source: §3.1 · p. 2

3.2 Finding Activation Patterns

  • Tokens exceeding 70% of a feature’s maximum activation are collected from the evaluation set to characterize its activation patterns.The threshold is based on each feature’s maximum activation.Source: §3.2 · p. 3

3.3 Language Selectivity Metrics

  • Features are classified as English or Japanese when the corresponding language probability exceeds 0.9; otherwise, they are classified as mixed.The probabilities are conditioned on the feature being activated and sum to one.Source: §3.3 · p. 3, §3.2 · p. 3

3.4 Concept Selectivity Metrics

  • Semantic Entropy measures the diversity of meanings among activating tokens by embedding them, clustering by cosine similarity, and computing entropy over the resulting clusters.The cosine-similarity threshold is 0.1; token embeddings come from the 3.7B model’s embedding layer.Source: §3.4 · p. 3
  • At layer 14 of the 3.7B model, mixed-language features decrease as monosemanticity rises during intermediate training, then mixed features rise while monosemanticity remains high late in training.The figure describes early training as having many mixed features and low monosemanticity.Source: Figure 3 · p. 4
  • Monosemanticity is the complement of semantic entropy divided by token entropy; values near 1 indicate varied tokens with similar meanings, and the single-token case is defined as 1.Token Entropy measures the diversity of tokens activating a feature.Source: §3.4 · pp. 3–4

4 Observations

The observations trace a shift from initially unstructured or language-specific features toward bilingual alignment, most evident in mid-layers and stronger in larger models.

4.1 LLMs first learn languages independently before aligning them bilingually

  • By late training, features activate on semantically related Japanese and English tokens, following earlier language-specific feature learning.Early features show random activation; in the middle phase, features become language-specific and more monosemantic.Source: §4.1 · pp. 4–5, Figure 4 · p. 5

4.2 Mid-layers capture more bilingual alignments

  • Lower layers become more polysemantic, with features spanning multiple meanings across tokens.The paper attributes this to proximity to the input, where the model must distinguish about 100,000 vocabulary tokens—more than the SAE intermediate-layer dimension.Source: §4.2 · pp. 5–6
  • Upper layers increasingly activate on longer, contextually connected spans rather than individual tokens.Their mixed-language proportion stays below the mid-layers, while monosemanticity declines as training progresses.Source: §4.2 · p. 6
  • Mid-layers specialize in bilingual representations, balancing monosemanticity with mixed-language proportion as layer behaviors diverge later in training.The mid-layers follow the 14th-layer pattern, while lower and upper layers develop distinct patterns.Source: §4.2 · pp. 5–6

4.3 Larger LMs develop more bilingual alignments

  • Larger models show a greater ability to learn bilingual features in the mid-layers, while smaller models struggle.In the smaller model, mid-layer mixed-language proportion rises less and features are less inclined to show high monosemanticity across languages.Source: §4.3 · pp. 6–7
  • Smaller models show weaker bilingual development and also struggle to capture semantics within languages or context-level meanings.The study reports relatively high mixed-language proportion and low monosemanticity in their lower layers, and smaller span-length increases in their upper layers.Source: §4.3 · p. 7

5 Intervention

The intervention selectively injects bilingual features from a fully trained model into a mid-training model to test their contribution to performance. Bilingual representations improve both languages, while the method depends on access to the fully trained model’s layer output.

5.1 Method

  • The method decomposes fully trained representations into English-specific, Japanese-specific, and bilingual features, then selectively injects them into a mid-training model.The intervention uses a binary mask to activate bilingual features and sets its strength α to 0.1 in the reported experiments.Source: §5 · p. 7, §5.1 · p. 7

5.2 Setup

  • Experiments used the 14th layer of a 3.7B model and a checkpoint at 10,000 steps, approximately 40B training tokens, with 5,000 selected feature dimensions.Each setting was evaluated five times and results were averaged.Source: §5.2 · p. 7

5.3 Results & Discussion

  • Adding bilingual representations improved both languages’ performance and reduced perplexity more than adding English- or Japanese-specific representations.The comparison uses the mid-training model without intervention as the baseline.Source: §5.3 · p. 7, Table 1 · p. 8
  • The intervention requires the fully trained model’s Layer ℓ output, so an SAE alone cannot directly enhance a mid-training model.The authors suggest that acquiring bilingual knowledge in later pre-training stages could benefit performance.Source: §5.3 · p. 8

6 Related Work

Prior work uses SAEs to interpret features across layers, architectures, and training stages, while multilingual studies examine shared and language-specific neurons. This study investigates bilingual feature formation across training stages, sizes, and layers, and intervenes on those representations.

6 Related Work

  • SAE research has expanded from single-model feature decomposition to comparisons across layers, architectures, and fine-tuning stages.The cited concurrent work tracking feature formation during training lacked quantitative evaluation.Source: §6 · p. 8
  • Multilingual neuron studies report both neurons shared across languages and tasks and language-specific neurons linked to model performance.Prior analysis also found increasing multilingual understanding with larger models and continued training, aligning with this study’s SAE analysis.Source: §6 · p. 8
  • The study contributes a comparative analysis of bilingual capability across training stages, model sizes, and layers, alongside direct SAE interventions on bilingual representations.Prior work tracked feature formation during training but lacked quantitative evaluation.Source: §6 · p. 8

7 Conclusion

The study traces bilingual representation development with SAEs and reports that languages are learned independently before later bilingual alignment, particularly in mid-layers of larger models. Targeted SAE interventions further demonstrate the importance of bilingual representations and the potential of SAEs for both analysis and intervention.

7 Conclusion

  • Bilingual language models initially learn languages independently and later develop bilingual alignments, particularly in the mid-layers of larger models.Source: §7 · p. 8
  • Targeted SAE interventions demonstrate the importance of bilingual representations in the studied models.Source: §7 · p. 8
  • The study uses SAEs to manipulate internal representations as well as interpret them.Source: §7 · p. 8

8 Limitations

The study examines English and Japanese, so its findings may not generalize to all language pairs. Its SAE analysis also faces interpretability concerns and imperfect reconstruction, making the representation analysis approximate.

8 Limitations

  • Because the study focuses on English and Japanese, its findings may not generalize to all language pairs.Source: §8 · p. 8
  • Concerns about SAE interpretability limit confidence in interpreting the features used in the analysis.Source: §8 · p. 8
  • Imperfect SAE reconstruction means the analysis approximates the model’s internal representations.Source: §8 · p. 8

A Training Details

SAE learning rates were selected by validation-set reconstruction loss after a grid search at the last checkpoint and middle layer. The study reports that smaller learning rates worked better for larger language models.

A.1 Learning Rate Selection

  • The learning-rate grid search tested six rates for each model size at the last checkpoint and middle layer.The selected rate minimized validation-set reconstruction loss.Source: §A.1 · p. 11
  • Figure 11 plots learning rate against SAE reconstruction loss across model sizes, with stars marking each model’s lowest-loss point.Source: Figure 11 · p. 11
  • Smaller learning rates were more effective for training SAEs on larger language models.The selected learning rates are summarized in Table 2, but their values are not provided here.Source: §A.1 · p. 11, Table 2 · p. 11

A.2 Time for training SAEs & the number of stored activations

The section points to Table 3 for further details.

A.2 Time for training SAEs & the number of stored activations

B Ablation Study of Adding Bilingual Features

The appendix presents an ablation of different alpha and m values and includes figures tracking language mixing, monosemanticity, and span length across training stages and model sizes. Table 4 identifies the mid-training model without intervention as the perplexity baseline.

B Ablation Study of Adding Bilingual Features

Loading 2503.06394v2…