PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Multilingual Language Models by Aligning Representations through Steering

Omar Mahmoud, Buddhika Laknath Semage, Thommen George Karimpanal, Santu Rana

arXiv:2505.12584v2cs.CL

TL;DR

How LLMs internally represent non-English inputs remains underexplored, despite translation-based approaches being constrained by quality and cost. This paper steers representations with a learned vector at one layer, reporting results above NLLB and competitive with Google Translate across multiple datasets.

  • Problem

    How LLMs internally process non-English inputs remains underexplored, while translation-based approaches are limited by translation quality and cost.

    Source: §1 · p. 1

  • Method

    The paper learns an English-alignment steering vector and uses it to steer representations at a single model layer.

    Source: §1 · p. 2

  • Results

    The method surpasses the NLLB translation baseline and achieves results competitive with Google Translate across multiple datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Steering achieves competitive performance at a fraction of the computational and operational cost of supervised fine-tuning.

    Source: §4.2 · p. 5

    The fixed linear steering direction may not capture intricate language mappings fully, and experiments isolate one layer and language at a time.

    Source: Limitations · p. 9

Abstract

from arXiv · show

This paper investigates how Large Language Models (LLMs) represent non-English tokens -- a question that remains underexplored despite recent progress. We propose a lightweight intervention method using representation steering, where a learned vector is added to the residual stream at a single model layer to enhance multilingual performance. Through extensive experiments across seven competitive baselines -- including prompt optimization, supervised fine-tuning (SFT), in-context learning, cross-lingual transfer, and translation-based methods-we show that our approach consistently outperforms most alternatives. In particular, it achieves performance on par with production-grade translation systems while requiring far fewer resources. We further explore the complementarity between our method and SFT, demonstrating that steering offers a direct, efficient way to realign internal representations. These findings underscore the potential of activation-level interventions as a powerful tool for improving the multilingual capabilities of LLMs.

1 Introduction

LLMs remain predominantly English-focused, and how they internally process multilingual inputs is still poorly understood. The paper proposes single-layer representation steering and reports improved multilingual performance, including results above NLLB and competitive with Google Translate.

1 Introduction

  • Non-English and especially low-resource languages are underrepresented in LLM advances, while translation-based processing is limited by translation quality and cost.The introduction argues for multilingual capability within models rather than relying solely on translation.Source: §1 · p. 1
  • The paper asks whether multilingual alignment can be achieved more efficiently by steering one layer rather than modifying representations across all layers.Prior work used least-squares steering across all layers; this paper proposes a single-layer intervention without fine-tuning.Source: §1 · p. 1
  • A single steering vector can be shared across structurally similar languages, enabling zero-shot cross-linguistic transfer without language-specific fine-tuning.The paper presents this as a contribution alongside its single-layer alignment method.Source: §1 · p. 2
  • Single-layer steering significantly surpasses the NLLB translation baseline and achieves results competitive with Google Translate across multiple datasets.The method learns an alignment with English and applies it to representations at one layer without fine-tuning.Source: §1 · p. 2

2 Related Work

Prior multilingual work largely develops capabilities through training, prompting, or external modules, while comparatively few studies examine how LLMs internally manage multilingualism. Representation engineering and inference-time steering offer relevant tools, but multilingual applications remain underexplored.

2 Related Work

  • Multilingual advances mainly use cross-lingual instruction fine-tuning, prompt strategies, or external modules, while fewer studies investigate internal multilingual processing.The passage contrasts external support such as LangBridge and representation alignment such as MindMerger with studies of internal mechanisms.Source: §2 · p. 2
  • Representation engineering studies how concepts are processed inside LLMs and has been applied to alignment, model editing, and vulnerability analysis.The passage cites Zou et al. (2023) for representation engineering, Wang and Shu (2024) and Li et al. (2024) for vulnerabilities, and Lu and Rimsky (2024) for concept representations.Source: §2 · p. 2
  • Steering vectors can manipulate internal states at inference time, but their use for multilingual settings remains largely unexplored.Prior examples modify residual-stream or hidden-state activations for truthfulness and refusal behaviors.Source: §2 · p. 2

3 Background

The background motivates steering by showing gains from self-translation while noting that non-English prompts still lag English. The proposed approach learns a language-alignment vector and applies it to target-language residual activations at one layer.

3.1 Evaluating LLM’s capabilities

  • Self-translation improves Llama2’s performance by 2.4% on average, but non-English prompts still do not reach English-level understanding.The passage also reports slight low-resource-language improvements for Aya23 and attributes the remaining gap likely to representation-mapping limitations.Source: §3.1 · p. 2, Table 1 · p. 3
  • The method learns a steering vector from English and target-language corpora at a specific layer, then applies it to each prompt token’s residual stream there.The overview presents DPO or MSE as the vector-learning options.Source: Figure 1 · p. 3
  • A transformer maps input-token sequences to output probability distributions, with residual activations initialized from embeddings and updated by attention and MLP components.After the layers, final logits are unembedded and passed through softmax to produce output distributions.Source: §3.1 · p. 3
  • DPO learns an alignment direction by preferring English-aligned target representations while disaligning original target activations.This differs from mean-difference or PCA directions by optimizing a bidirectional preference relationship.Source: §3.1 · p. 4
  • The alternative loss-based method learns the vector by minimizing mean squared error between aligned target-language and English representations.The approach follows work suggesting representations across languages may be linearly mappable.Source: §3.1 · p. 4
  • At inference, the learned vector modifies target-language residual activations at layer L, and the altered activation flows through the remaining layers.The method aims to make target-language representations more English-like and observe the effect on output distributions.Source: §3.1 · pp. 3–4

4 Experiments

The experiments evaluate multilingual steering across several open-source models, task datasets, and comparison methods. DPO steering improves over several baselines, including NLLB, while approaching Google Translate.

4.1 Experimental Setup

  • The evaluation included five open-source models, with the main discussion focusing on LLama2-7B Chat and Aya23-8B.Comparisons included basic prompting, translation to English, XLT, 5-shot learning, and SFT.Source: §4.1 · pp. 4–5
  • Steering vectors were trained using MSVAMP across 14 languages and 1,000 Tatoeba instances per language across 50+ languages, then evaluated on five task datasets.The evaluation covers mathematical reasoning, natural language inference, causal commonsense, general knowledge, and comprehensive language understanding.Source: §4.1 · p. 4

4.2 Results

  • DPO improves 5-shot learning by 38.8%, SFT by 26.7%, and NLLB by 25.4%, while trailing Google Translate by 3.08%.The authors report improvements over these baselines and a narrow gap to Google Translate.Source: §4.2 · p. 5
  • DPO and MSE steering improve over standard prompting and XLT; steering is described as scalable and less costly than SFT.The authors characterize steering as achieving competitive performance at a fraction of SFT's computational and operational cost.Source: §4.2 · p. 5
  • DPO consistently outperforms MSE steering; the authors attribute this to DPO's directional optimization signal and report larger gains as model size increases.They contrast DPO's directional guidance with MSE's measurement of error magnitude without gradient directionality.Source: §4.2 · p. 5

5 Analysis

Section 5 analyzes internal language representations, steering-vector generalization, cross-language transfer, and the prominence of high-resource languages. It links representation alignment to multilingual performance and identifies limits on steering and the scope of current models’ reasoning capabilities.

5.1 Can we quantify the quality of the internal translation process?

  • Internal representation alignment tracks pre-training data volume, and weaker alignment is described as a direct cause of performance disparities across language groups.LLaMA2 aligns strongly for French and German but weakly for Thai and Telugu; Figure 2’s 0.5 threshold separates high-resource from mid-/low-resource languages.Source: §5.1 · pp. 5–6, Figure 2 · p. 6

5.2 Challenges in Steering Vector Generalization

  • Steering is distribution-sensitive, with performance dropping significantly on out-of-distribution tasks, and its static correction misses prompt-specific variation.The paper points to context-aware steering that adapts dynamically to each prompt as a future direction.Source: §5.2 · p. 6

5.3 Is the steering vector transferable across languages?

  • Steering-vector transfer generally improves performance within related language families but is ineffective across distant families such as Spanish to Japanese.Transfers from Spanish to German, French, and Russian, and between Japanese and Chinese, are reported as effective.Source: §5.3 · p. 6

5.4 High Resource Languages are dominant in Representation Space

  • High-resource languages including Spanish, German, and French perform on MGSM at levels close to English, while current models’ reasoning support remains concentrated in a privileged high-resource set.The paper frames extending that set to low-resource languages as the central challenge for multilingual modeling.Source: §5.4 · p. 7

6 Ablation Studies

The best injection layer depends on model architecture and language-resource level: early steering benefits Aya23, while LLaMA2’s response varies by language group. Steering English prompts can also have a negative impact, especially when representational distance is greater.

6.1 Impact of Injection Across Model Layers

  • Aya23 benefits most from early-layer steering, while LLaMA2 benefits from early and middle layers for low-resource languages but can be disrupted for high-resource languages.For both models, steering the final layers yields little to no improvement.Source: §6.1 · p. 7

6.2 Impact of Steering Vectors on English Capabilities

  • English-prompt probing has a negative impact that intensifies as the steering vector’s language becomes more representationally distant from English.The relationship is pronounced in LLaMA2 and slightly reduced in Aya-23.Source: §6.2 · p. 7

7 Fine-tuning vs. Steering Approach

The section compares fine-tuning and steering through logit-lens analysis of internal alignment with English. Both methods improve alignment, but steering makes a targeted correction immediately, while fine-tuning does so gradually.

7 Fine-tuning vs. Steering Approach

  • The logit lens projects a model’s post-activation state into vocabulary space, then uses softmax probabilities and language detection to assess alignment with English.The analysis takes the state at layer L + 1 and examines likely tokens.Source: §7 · pp. 7–8
  • Table 3 describes English as the most dominant language in the representation space, with French, Spanish, and German achieving comparable results.These high-resource languages were selected as agnostic languages in that space.Source: Table 3 · p. 8
  • In Table 4’s MGSM results for Llama2-7B and Aya23-8B, the steering vector is reported to have a negative impact on English prompts.Source: Table 4 · p. 8
  • The base model shows weak cross-lingual alignment, while the fine-tuned model learns alignment after extensive translation fine-tuning.The steered model matches that alignment instantly using one steering vector.Source: §7 · p. 8
  • Both fine-tuning and steering improve alignment with English, but steering provides an immediate targeted correction while fine-tuning aligns representations gradually.The authors also report that advanced multilingual models already show strong English alignment without intervention, linking the need for steering to limitations in base-model training.Source: §7 · p. 8

8 Conclusions

The conclusion reports that probing inner layers boosts multilingual task performance but may hinder monolingual performance. It also emphasizes that model families respond differently to layer-level changes.

8 Conclusions

  • Probing inner layers boosts multilingual task performance but may hinder monolingual performance.The conclusion frames this alongside comparisons of SFT, RLHF, steering, and probing, while identifying limitations in steering vectors’ handling of linguistic nuances.Source: §8 · p. 8
  • LLM families differ in their sensitivity to layer-level changes, making careful tuning and alignment important to optimizing multilingual performance.Source: §8 · p. 8
  • The paper examines multilingual processing across languages with varying resource levels.Source: §8 · p. 8

Limitations

The authors identify scalability and representational limits in their steering approach, and note that their experiments isolate one layer and one language at a time.

Limitations

  • Sweeping across all model layers for probing is not scalable for LLMs and is impractical for real-world applications.Source: Limitations · p. 9
  • A fixed linear steering direction cannot fully capture intricate mappings between languages; the authors see token-specific vectors as a more promising direction.Source: Limitations · p. 9
  • The experiments isolate one layer and one language at a time, whereas a more powerful system would handle multiple languages and layers through multi-objective training.The authors describe the isolation as necessary to establish a clear baseline.Source: Limitations · p. 9

Ethics Statement

The paper acknowledges risks and limitations associated with multilingual processing and steering. It describes learning a steering vector through language-behavior optimization and gives algorithm excerpts for extracting activations, injecting the vector, calculating losses, and updating it.

Ethics Statement

  • The ethics statement acknowledges risks from language-representation bias, unequal performance across resource levels, and unintended consequences of steering.It also notes possible inherited biases from training data or model architectures and calls for responsible deployment.Source: Ethics Statement · p. 9

A Learning the Steering Vector

  • The steering vector is optimized to increase responses matching a desired language behavior and reduce responses matching the opposite behavior.The bidirectional objective uses a directional coefficient to select English behavior or its opposite.Source: A Learning the Steering Vector · p. 12

A.1 Algorithms

  • The algorithm excerpts describe extracting layer-L activations, injecting the steering vector, computing a loss, and updating the vector.The excerpts identify both MSE and DPO loss procedures.Source: §A.1 · pp. 12–13

A.2 Other learning methods

The section reports learnable steering’s advantage over PCA and mean-difference approaches, then relates steering to SFT and examines multilingual alignment, larger-model trends, and task examples.

A.2 Other learning methods

  • Learnable steering vectors surpass PCA and mean-difference approaches across all tasks on Aya32-8B and LLama2-7B.BiPO’s dynamic feedback loop is described as capturing multilingual features more effectively than PCA or mean-difference activation shifts.Source: Figure 4 · p. 13, §A.2 · p. 13

B SFT Vs Steering: Problem Setup and Notation

  • SFT models layer transformations using instruction pairs, while steering learns one shared vector applied across question-answer pairs.Steering modifies activations with a learned direction whose scaling coefficient controls its magnitude.Source: B SFT Vs Steering: Problem Setup and… · p. 14

C Hyperparameters

  • Steering used AdamW, a 5×10−4 learning rate, 0.05 weight decay, batch size 1, cosine scheduling, and one epoch, except some languages used three.For MSE, the learning rate was 1 × 10−8 with 3, 5, 8, or 12 epochs; SFT used five epochs, a 1 × 10−3 learning rate, batch size 16, and 0.001 weight decay.Source: C Hyperparameters · p. 14

C.1 High-Capability Models and Inner Translation Behavior

  • High-capacity models map multilingual tokens into English-aligned representations early, with minimal inner-translation loss; SFT and RLHF are associated with stronger alignment.Aya-Expanse’s alignment is attributed to techniques including RLHF.Source: §C.1 · pp. 14–15

D Larger LLMs Exhibit Consistent Behavior

E Examples of different tasks

  • In the examples, steering gives a $106 loan total without an extraneous monthly-payment calculation, but returns $18 rather than the original answer’s $26 for egg sales.Source: E Examples of different tasks · p. 20
Loading 2505.12584v2…