Source-linked AI summary
BabelSteering: Multilingual Safety Alignment via English Steering Vectors
Emma V. Stein, Dominik Meier, Terry Ruas, Jan Philip Wahle, Bela Gipp
TL;DR
Multilingual safety research and alignment remain limited relative to English, leaving uncertainty about whether English-derived safety signals generalize across languages. BabelSteering applies English-derived refusal steering at inference time and increases harmful-prompt refusals across eight languages with negligible general-task performance impact, though over-refusal rises.
Problem
Safety research and alignment remain heavily concentrated on English, limiting evidence for robust safeguards in non-English languages, especially where multilingual safety data are costly to scale.
Method
BabelSteering combines English-derived refusal and false-refusal steering vectors into a lightweight inference-time intervention evaluated across eight languages and multiple safety and utility dimensions.
Results
11 pp average refusal-rate increase across languages for Gemma 7B, with no reduction on Global MMLU and increased pseudo-harmful refusal from 9% to 21%.
Takeaways & Limitations
BabelSteering offers a cheap, low-data, tunable intervention for improving multilingual safety while preserving general cognitive performance, despite measurable over-refusal.
Takeaways & Limitations
Because BabelSteering reflects English-centric safety norms, it may overlook culture-specific harms and cannot replace culture- and language-specific safety research.
Abstract
from arXiv · showhide
Large language models (LLMs) are deployed globally in high-stakes settings, yet most safety research and alignment efforts remain concentrated on English. Thus, users interacting with LLMs in other languages may encounter weaker safeguards despite relying on the same systems for similarly sensitive tasks. In this work, we investigate whether safety signals learned from a high-resource language, like English, can improve multilingual safety. We propose BabelSteering, an activation steering method that acts as a lightweight inference- time intervention, using refusal directions derived from English safety supervision to generalize across languages. Our evaluation includes eight languages and jointly measures refusal of harmful requests, over-refusal, and general task utility. The results show that BabelSteering increases the refusal of harmful requests across languages, with only a marginal to no reduction in task utility but with some increase in refusal of pseudo-harmful prompts. For example, for Gemma 7B, we see an average increase in the refusal of harmful prompts across languages of 11 percentage points (pp), with individual languages like Bengali seeing an increase of 17 pp, with no loss of utility on Global MMLU, while pseudo-harmful refusals increase by 13 pp on average. We also introduce a multilingual translation-and-evaluation pipeline to facilitate future work on cross-lingual safety interventions. Overall, our findings suggest that activation steering may provide a practical, low- cost mechanism for extending English-derived safety signals to other languages. Warning: this paper contains examples with unsafe content
1 The Multilingual Safety Gap
LLM safety research and alignment remain heavily concentrated on English despite global deployment, creating a multilingual safety gap that is costly to address with language-specific data and retraining. BabelSteering tests whether English-derived refusal and overcaution directions can transfer across eight languages while preserving utility.
- Motivation: Safety research remains heavily concentrated on English, even as LLMs serve billions of users across many languages.The second most studied language, Chinese, has around 10 times fewer safety research papers than English.
- Motivation: Language-specific safety alignment is expensive to scale, especially for low-resource languages with limited access to first-language speakers.Current pipelines rely on carefully annotated training data and high-quality user feedback on response safety.
- Approach: BabelSteering transfers English-derived refusal and overcaution vectors across eight languages using a translation-and-evaluation pipeline spanning mid- and low-resource settings.The method uses orthogonalized activation steering and English safe and pseudo-harmful data without language-specific retraining.
- Results: 11 pp average increase in refusal rates across languages for Gemma 7B occurred with no reduction on Global MMLU, while pseudo-harmful refusal rose to 21% from a 9% baseline.The evaluation jointly measures refusal, over-refusal, and general task performance across eight languages.
- Results: Pseudo-harmful refusal increased by 4 pp for Chinese but by 23 pp for Bengali, with larger cross-language differences generally appearing in low-resource languages.These languages differ in how well refusal behavior is already learned.
- Implications: The pipeline supports future cross-lingual safety testing, while steering improves safety without or with only marginal utility loss and requires no inference-time training or additional resources.The authors report that a small amount of highly available English data is sufficient.
2 Prior Work on Multilingual Safety
Prior multilingual safety work is constrained by English-dominated data and limited non-English evaluation, while existing interventions incur scalability, efficiency, translation-quality, or training risks. BabelSteering addresses these limitations through lightweight, weight-update-free activation steering based on English contrastive data, building on mechanistic evidence for refusal directions.
- Motivation: English-dominated pre-training and scarce non-English evaluation contribute to multilingual performance and safety disparities, especially for low-resource languages.Ahuja et al. (2024) linked data imbalance directly to downstream performance, creating barriers where high-quality data is difficult to obtain.
- Limitations of prior interventions: Existing multilingual interventions face scalability and efficiency trade-offs, including computationally intensive fine-tuning on generated and translated data.Translation quality and training on models’ own outputs introduce additional risks.
- BabelSteering: BabelSteering requires no weight updates and intervenes directly in activation space using a small set of high-quality English contrastive pairs.The approach is designed as a computationally lightweight alternative for robust multilingual refusal and can be adapted through steering strength.
- Mechanistic basis: Recent mechanistic interpretability work motivates activation-level safety interventions by identifying refusal behavior as often mediated by a single residual-stream direction.Wang et al. (2024) further used orthogonalized true- and over-refusal vectors to reduce false refusals in English settings.
3 BabelSteering
BabelSteering is a lightweight inference-time intervention that combines English-derived refusal and false-refusal directions to improve multilingual safety while limiting over-refusal. It evaluates safety, over-refusal, and utility across three model families and eight languages using multilingual benchmarks and classifier-based refusal rates.
- Method: Steering modifies internal representations at inference time without changing model weights or requiring additional training.The method adapts prior English steering-vector extraction for multilingual safety and over-refusal control.
- Method: The refusal direction uses difference-in-means activations from harmful and harmless prompt sets, with layer selection governed by a KL-divergence threshold of 0.1.The English dataset used for extraction contains n=128 examples, and coefficient α controls intervention strength.
- Method: The intervention adds a refusal vector at one selected layer and ablates an orthogonalized false-refusal vector across all layers.The refusal vector increases harmful-prompt refusals, while false-refusal ablation counteracts refusals on benign inputs.
- Evaluation: Evaluation measures safety, over-refusal, and general task performance across Gemma 7B, Llama 3.1, and Qwen 7B.Safety uses MultiJail; over-refusal uses a translated OR-Bench subset; utility uses the human-translated Global MMLU (Lite) subset.
- Evaluation: Eight evaluated languages are English, Chinese, Italian, Vietnamese, Arabic, Korean, Thai, and Bengali, spanning diverse scripts and resource levels.MultiJail provides 315 parallel harmful prompts per language across ten languages, while the translated over-refusal dataset was released publicly.
- Evaluation: Refusal rates for MultiJail and OR-Bench are classified with WildGuard, while Global MMLU performance is reported as accuracy.The authors avoid substring matching because refusal phrasing varies across languages and may not survive translation.
4 Experiments
BabelSteering reduces harmful-prompt answering across languages and models, while preserving general task performance largely unchanged. However, stronger refusal steering increases pseudo-harmful over-refusal, making joint safety–compliance evaluation and empirical hyperparameter validation necessary.
- Harmful-request refusal: 9 pp in Chinese, 12 pp in Arabic, and 17 pp in Bengali: Gemma’s harmful-prompt answering rate decreases across high-, mid-, and low-resource languages.Qwen shows the largest decrease, Llama 3.1B the smallest, and English changes by 3pp.
- Harmful-request refusal: Higher-quality evaluation-time translations narrow the observed safety gap between high- and low-resource languages relative to prior literature.The authors warn that translation quality can introduce noise into multilingual safety interpretations and emphasize specifying the exact variety used.
- Hyperparameters and model variation: λ ≤0.8 and α ≤0.8 strike the best balance between safety and compliance, although the optimal tradeoff depends heavily on the usage scenario.Increasing λ and α generally makes models more cautious in both safety and compliance dimensions, with exceptions across model settings.
- General task utility: Less than 3pp: increasing steering intensity affects MMLU performance far less than Multijail or OR-Bench, while general language-understanding performance remains largely unaffected.The authors note that Global MMLU provides a consistent baseline but does not fully capture culture-specific human preferences.
- Overall findings: BabelSteering increases safety across Bengali and Arabic while introducing measurable over-refusal and negligible impact on general cognitive performance.The authors characterize the intervention as cheap, low-data, and tunable, with safety concepts represented consistently enough for English-derived vectors to transfer cross-lingually.
5 Epilogue
BabelSteering transfers English refusal behaviors to other languages, improving multilingual safety with minimal general-utility impact but some increased caution. The paper emphasizes holistic evaluation that includes refusal, over-refusal, helpfulness, and general utility.
- Motivation: BabelSteering addresses the multilingual safety gap caused by safety research and deployment practices disproportionately centered on English.The work aims to improve safety across languages and include marginalized groups.
- Evaluation: Holistic evaluation should consider refusal behavior, over-refusal of harmless or pseudo-harmful prompts, helpfulness, and general model utility.The appropriate weighting of these factors depends on the use case.
- Findings: Up to 32 pp: BabelSteering increases refusal of unsafe prompts across languages, with minimal impact on general utility but sometimes making models overly cautious.The method transfers well-studied English refusal behaviors to other languages.
- Future Work: The multilingual translation-and-evaluation pipeline supports evaluation of future multilingual safety interventions.The authors present the framework as a foundation for future research.
6 Limitations
BabelSteering is not a panacea: it reflects English-centric safety norms and may overlook nuanced, culture-specific concepts of harm. The authors therefore frame it as a complement to, not a replacement for, culture- and language-specific approaches, while noting that the pipeline uses machine translation because human translation is costly.
- The method inherently reflects English-centric safety norms and may overlook nuanced, culture-specific concepts of harm.
- BabelSteering is a complement, not a replacement, for culture- and language-specific approaches to safe models.
- The pipeline uses machine translation rather than human translation because of the associated cost.The authors report selecting high-quality machine-translation models.
Ethical Considerations · A Appendix
BabelSteering is intended to extend safety protections to underserved non-English-speaking communities, but it may propagate English-centric safety norms and reduce incentives for culture-specific research addressing localized harms.
- Ethical Considerations: BabelSteering aims to extend safety protections to underserved non-English-speaking communities.
- Ethical Considerations: The method risks propagating English-centric, specifically Western or American, safety norms.
- Ethical Considerations: These English-centric safety norms may not align with diverse cultural contexts.
- Ethical Considerations: Because BabelSteering cheaply increases safety across languages, it could disincentivize resource-intensive, culture-specific research.
- Ethical Considerations: Such culture-specific research is necessary to address localized harms.
- A Appendix: The authors frame BabelSteering as an immediate intervention to raise safety.
A.1 Hyperparameter Runs across all tested values and Models.
Across benchmark runs and tested models, increasing either α or γ generally raises refusal and over-refusal while slightly reducing utility.
- Increasing α generally increases refusal.
- Increasing γ generally increases over-refusal.
- Increasing either α or γ slightly decreases utility.
A.2 Per Model and Language Tables
The section presents detailed model-performance tables for each language and reports that the intervention has a larger impact on lower-resource languages.
- Per-Model and Language Results: The tables detail model performance separately for each language.They provide per-language results for the evaluated models.
- Per-Model and Language Results: The intervention’s impact is larger on lower-resource languages.This is reported as an overall trend in the per-language results.
- Per-Model and Language Results: The section organizes the results by both model and language.Its purpose is to show detailed performance across these dimensions.
C AI Use
The research used Gemini 3.1 Pro Preview, GPT-5.3 Chat, and Claude Sonnet 4.6 to assist with programming and writing. The authors acknowledge these tools’ limitations and describe their use as transparent and responsible.
- C AI Use: AI tools assisted the research team with programming and writing, including phrasing and grammar.The named tools were Gemini 3.1 Pro Preview, GPT-5.3 Chat, and Claude Sonnet 4.6.
- C AI Use: The authors note that the AI tools have inherent limitations.
- C AI Use: The authors state that they used AI transparently and responsibly.They also describe the conclusions as resulting from combined human involvement, though the supplied passage is truncated.
D Experimental Configuration · CITATION SHEET · BibTeX Entry
The experiments used Gemma 7B with λ =0,8 and α=0,4, running on a single 80GB A100 for roughly 100 GPU hours. The supplied block also contains citation-sheet generation information and a BibTeX entry for the 2026 paper.
- D Experimental Configuration: Gemma 7B experiments used λ =0,8 and α=0,4 on a single A100 with 80GB of VRAM, requiring roughly 100 GPU hours.The reported compute covered experimentation and evaluation.
- D Experimental Configuration: Table 8 reports Llama-3.1 results using MultiJail, OR-Bench, and MMLU percentage metrics.The passage identifies the table’s model and metric set but provides no numerical values.
- D Experimental Configuration: Table 9 reports Qwen-7B results using MultiJail, OR-Bench, and MMLU percentage metrics.The passage identifies the table’s model and metric set but provides no numerical values.
- CITATION SHEET: The citation sheet was generated with citeassist.uni-goettingen.de.This is the only information supplied for the citation-sheet subsection.
- BibTeX Entry: The BibTeX entry lists the authors as Emma V Stein, Dominik Meier, Terry Ruas, Jan Philip Wahle, and Jan Gipp.The supplied author field is truncated after “Gipp,” but the visible names are included here.
- BibTeX Entry: The document states that the BibTeX entry was generated on August 18, 2026.This date appears as a separate generated-date passage.