PaperlayerView exact revision on arXiv

Source-linked AI summary

When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMs

Apoorva Upadhyaya, Sandipan Sikdar

arXiv:2608.29936v1cs.CLcs.LG

TL;DR

Multilingual safety alignment degrades outside English, but its internal mechanism remains poorly understood. The paper uses layer-resolved SAE features across three instruction-tuned LLMs and eight languages to analyze safety-language organization and interventions. It finds architecture-dependent entanglement that predicts safety interventions’ language-identity and fluency costs.

  • Problem

    The internal mechanism behind uneven multilingual safety alignment remains poorly understood.

  • Method

    The study analyzes harm, harmless, and language identity SAE features across three instruction-tuned LLMs, eight languages, and all layers.

  • Results

    Safety-language entanglement is architecture-dependent and predicts harmful-response, language-identity, and fluency costs during safety interventions.

  • Takeaways & Limitations

    The findings provide a mechanistic account of multilingual safety interventions and motivate geometry-aware evaluation of cross-lingual safety transfer.

  • Takeaways & Limitations

    The findings may not generalize beyond three model families, eight languages, or the top-k k=5 MI-based feature selection.

Abstract

from arXiv · show

Safety alignment of large language models (LLMs) degrades across languages, yet the internal mechanism driving this asymmetry remains poorly understood. Our work, therefore, presents a systematic mechanistic analysis of multilingual safety using sparse autoencoder (SAE) features, sparse interpretable directions in the residual stream associated with harmful and harmless model behavior across three instruction-tuned LLMs, eight languages, and all model layers. We observe that safety-relevant features are architecture-dependent in terms of where they are located and how they are distributed across layers. Additionally, they are geometrically entangled with language identity and exhibit cross-lingual sharing patterns, i.e., languages share safety features to varying degrees across model depths and architectures. This safety-language entanglement has direct consequences such that ablating safety features impacts not only harmful response rates but also target language, with the degree of intervention predicted by the relationship between safety and language features. Our findings qualify the language-universality of safety alignment as architecture-dependent and offer a mechanistic account of multilingual safety interventions.

1 Introduction

Multilingual safety alignment remains internally poorly understood despite growing benchmarks, training, and weight-editing approaches. This paper analyzes safety features and language identity jointly across architectures to explain intervention costs.

  • Safety alignment transfers unevenly across languages, with models producing more harmful responses outside English.
  • Existing mechanistic analyses do not reveal the geometric relationship between safety features and language identity or its architectural transferability.
  • The study analyzes harm, harmless, and language identity SAE features across three instruction-tuned LLMs, eight languages, and all layers.
  • Safety signals concentrate in late layers, but the ordering of harm and harmless features varies by architecture and language.
  • Safety-language entanglement is architecture-dependent, varies across depth, and predicts the cost of cross-lingual safety interventions.

2 Methods

The method extracts residual-stream activations, represents them with sparse autoencoder features, and selects safety- and language-associated directions using signed mutual-information scores. Controlled feature interventions then test effects on safety, language identity, and fluency.

  • Sparse Autoencoders (SAEs): SAEs encode residual activations into higher-dimensional sparse representations and reconstruct the original activations from learned feature directions.
  • Activation and feature extraction: The pipeline labels generated responses as harmful or harmless and associates each prompt with its input language.
  • Activation and feature extraction: For each language and layer, the method extracts the prompt’s final-token residual activation and projects it into SAE feature space.
  • Feature identification: Safety features are ranked by mutual information with response labels and signed by class-conditional activation differences, with positive scores indicating harmful association.
  • Feature identification: Language identity features use one-vs-rest binary mutual information and mean-activation differences to identify features associated with each language.
  • Intervention Strategies: Interventions modify the first T generated tokens by masking or amplifying selected SAE features, then decode the modified representations back into residual activations.

3 Experiments

Experiments span three open-weight instruction-tuned models, layer-wise SAEs, multilingual harmful-query data, and intervention metrics covering safety, language preservation, fluency, and feature entanglement.

  • Models and SAEs: The experiments use Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-IT, whose training and layer configurations differ.
  • Models and SAEs: Layer-wise SAEs cover all 32 Llama layers, 28 Qwen layers, and 42 Gemma layers.
  • Dataset and evaluation: TechHazardQA provides 7,745 potentially harmful or unethical questions, with 1,500 held out for intervention experiments.
  • Dataset and evaluation: Generated responses receive binary harmless-or-harmful judgments from an LLM-based majority-vote evaluator.
  • Metrics and interventions: The analysis measures identity-level feature overlap, geometric alignment, harmful response rates, language preservation, and fluency.
  • Metrics and interventions: Entanglement is measured per layer and across early, middle, and late phases using top-k k=5 features, Jaccard overlap, decoder cosine, and phase-relative z-scores.
  • Metrics and interventions: High decoder cosine with low Jaccard overlap indicates geometrically similar directions represented by different feature IDs.

4 Results and Analysis

Safety signals concentrate in late layers across architectures, but harm–harmless ordering, safety–language entanglement, and cross-lingual sharing vary by model and depth. Interventions confirm that decoder-cosine entanglement predicts language-identity and fluency costs.

  • RQ1: Near-zero early-layer MI across models and languages indicates safety computation is absent during initial surface-level feature extraction.Llama has isolated middle-layer peaks, whereas Qwen and Gemma defer safety signals entirely to late layers.
  • RQ1: Late-layer safety concentration is universal, but harm-first ordering appears in Llama and Qwen while Gemma more often exhibits harmless-first ordering.Both signals peak late in all models; harm precedes harmless in 5/8 Llama and 6/8 Qwen languages, while Gemma shows harmless-first ordering in 5/8.
  • RQ2: Jaccard overlap and decoder cosine reveal complementary entanglement because nonzero residual-stream alignment can persist when shared feature IDs are absent.Gemma has Jaccard = 0.000 with decoder cosine 0.077–0.129, and Qwen HI has Jaccard 0.090 with decoder cosine 0.570.
  • RQ2: Safety–language entanglement changes across depth and architecture: harm coupling rises in middle layers for Llama and Qwen, while late-layer entanglement peaks in Llama and Qwen but collapses in Gemma.Gemma shows six of eight languages with zero late-layer Jaccard despite nonzero decoder cosine.
  • RQ3: Decoder-cosine entanglement predicts intervention costs: harm ablation reduces harmful responses but causes larger language and fluency losses in Llama and Qwen than Gemma.For harm ablation, language entanglement is Llama 0.518, Qwen 0.541, and Gemma 0.041; Class 3 fluency is 76.1%, 79.3%, and 95.4%, respectively.
  • RQ3: Layer choice can reduce language cost: Llama harmless amplification at L18 reduces harmful responses by −12.14 pp with language cost ∆= −0.057 and Class 3 fluency of 92%.The intervention targets a clean zero-overlap window, consistent with low harmless–language entanglement.

5 Conclusion

The paper analyzes safety–language interaction with sparse autoencoder features across three architectures and eight languages. It finds architecture-dependent entanglement and cross-lingual sharing decoupled from safety depth, with entanglement predicting intervention costs before generation.

  • Conclusion: Sparse autoencoder features expose safety–language interaction across three instruction-tuned architectures and eight languages.The analysis spans model layers and examines interpretable safety and language-related directions.
  • Conclusion: Safety–language entanglement is architecture-dependent, while cross-lingual feature sharing is decoupled from safety signal depth.This qualifies the universality of multilingual safety organization across models.
  • Conclusion: Entanglement predicts the language-identity and fluency costs of safety interventions before generation, motivating geometry-aware evaluation of cross-lingual safety transfer.The conclusion frames this as guidance for more robust multilingual safety mechanisms.

6 Limitations

The study is limited by its coverage, feature-selection strategy, and intervention design, leaving several generalization and mechanistic questions open.

  • The evaluation covers three model families and eight languages, so findings may not generalize to other model sizes, alignment procedures, or languages.
  • Top-k (k=5) mutual-information feature selection is a coarse approximation of the full safety-relevant feature set.Entanglement patterns may vary with larger k or alternative selection criteria.
  • The experiments show entanglement predicts intervention cost but do not establish why entanglement differs across architectures.Possible drivers include pre-training data distribution, alignment procedure, and architectural design choices.
  • Interventions target top-5 features at a single peak layer, so multi-layer or larger-scale interventions may reveal richer safety-language trade-offs.

Ethical Considerations

The paper situates multilingual safety interventions within prior behavioral, SAE-based, and mechanistic work while highlighting misuse risks from manipulating safety features.

  • Manipulating identified internal features can increase harmful outputs for adversaries, motivating controlled experiments and withholding steering vectors and feature IDs.
  • Multilingual LLM Safety: Existing multilingual safety mitigations include consistency losses, reward-based transfer, and response-based distillation, but they implicitly assume safety representations are separable from language identity.
  • SAE-Based Interpretability: Prior SAE research analyzes language-specific representations and monolingual safety steering, jailbreak defense, neuron auditing, and refusal circuits.
  • Mechanistic Accounts: Mechanistic studies report cross-lingual refusal directions, shared safety neurons, and sparse weight editing for transferring English safety representations.
  • Mechanistic Accounts: The paper examines whether sparse weight editing remains separable from language identity, because high safety-language entanglement incurs language-identity costs.

B.3 SAE Quality Evaluation

The SAE quality evaluation measures reconstruction under out-of-distribution and in-distribution conditions, with high loss recovery indicating preservation of predictive behavior.

  • SAE reconstruction is evaluated under out-of-distribution WikiText-103 data and in-distribution held-out samples from the training blend.
  • The evaluation reports L0, variance explained, and loss recovered as mean ± standard deviation across all layers.
  • Values close to 1.0 indicate near-lossless reconstruction for the reported quality metrics.
  • High loss recovery under both settings confirms that SAE reconstructions preserve model predictive behavior across layers.
  • Human annotators were informed that translation and verification materials could contain unsafe or harmful content.
  • Translated TechHazardQA prompts achieved approximately 97% average agreement after review and proofreading across languages.

C.2 Dataset Annotation

The annotation setup uses judge-LLM prompts to classify generated responses for safety, with multiple evaluators and majority voting supporting cross-lingual labeling.

  • Three independent evaluator LLMs reduce dependence on any single judging model.
  • Figure 3 presents the prompt given to a judge LLM for evaluating whether generated responses are harmful or harmless.
  • Final harmful or harmless labels are determined by majority voting across the evaluator LLMs.
  • Judge labels agree with human annotations by approximately 94% on sampled harmful and harmless responses from each language.

D Per-Language Safety-Language Entanglement

Figures 10–15 compare phase-aggregated harm and harmless safety-language entanglement across languages and models, using raw similarities and phase-relative z-scores. The figures expose model- and signal-specific depth patterns, while cross-lingual matrices characterize pairwise sharing and English isolation.

  • Figure organization: Figures 10–15 organize each model by language rows and early, middle, and late columns, with harm and harmless features shown separately.Cells report raw values and phase-relative z-scores, where positive values are above the model’s layer average.
  • Layerwise safety signals: Harm and harmless MI signals are plotted across layers for Qwen and Gemma, with phase boundaries and peak layers marking their depth profiles.The shaded region highlights the gap between harm and harmless peaks.
  • Model-specific trajectories: Llama harm entanglement rises toward late layers, whereas harmless entanglement starts above average and declines below average by late layers.The late harm mean is +0.60, while harmless entanglement reaches z = −0.11 late.
  • Cross-lingual comparison: Cross-lingual similarity matrices report pairwise safety-feature similarity by phase and metric, with English consistently showing geometric isolation.The diagonal is masked, and each cell includes a raw similarity and phase-relative z-score.

F Orthogonal Feature Control Experiment

The orthogonal feature control experiment separates safety intervention from language-identity disruption. Features with minimal safety-language cosine preserve language identity while still modestly improving safety.

  • Control outcome: Orthogonal features preserve language identity with Δlang ≈0.01 while reducing harmful responses from 19.64% to 17.21%.This is a −2.43 pp change and a 12.4% relative reduction, compared with language collapse for entangled features.
  • Control outcome: Entangled features produce complete language collapse, with Δ = −0.613 alongside a 4.00 pp harmful-response reduction.The relative reduction is 20.4%, larger than with orthogonal features but at greater language cost.
  • Feature selection: The experiment selects five VI harmless and ZH harm features for minimal cosine to their respective language-identity features.The selected cosines are 0.029 for VI harmless and 0.038 for ZH harm, versus 0.451 and 0.466 for entangled features.

G Cross-Dataset Feature Transferability

The cross-dataset experiment applies harm features identified on TechHazardQA to Multi-Jail prompts without re-identification. Harmful responses decrease consistently, while language shift and fluency costs follow the features’ decoder-cosine entanglement.

  • Cross-dataset ablation: TechHazardQA top-5 harm features reduce harmful responses on Multi-Jail by −6.80 to −7.23 pp across four languages without re-identification.This tests transfer to another safety benchmark using the same features identified in RQ2.
  • Intervention cost: EN shows the lowest decoder cosine, minimal language shift of 0.039, and highest fluency of 92.60%.Its decoder cosine is 0.182.
  • Metric interpretation: Across models, decoder cosine exceeds Jaccard overlap, showing geometric entanglement beyond feature-ID sharing.Gemma reaches near-zero late-layer Jaccard for six of eight languages while decoder cosine remains non-zero.
  • Llama sharing patterns: Llama harm-feature sharing peaks in middle layers for most language pairs, with modest late-layer sharing and uniformly below-average EN pairs.Late-layer harm decoder-cosine sharing has a maximum of 0.166.
  • Llama sharing patterns: Llama harmless features retain more late-layer cross-lingual sharing than harm features, including decoder-cosine values of 0.275 for HI-ID and 0.271 for HI-VI.Late-layer harmless Jaccard is also higher than harm Jaccard for most pairs.
  • Qwen sharing patterns: Qwen reaches the study’s highest late-layer harm and harmless sharing, led by AR-RU harm cosine 0.684 and ID-VI harmless cosine 0.752.The corresponding Jaccard maxima are 0.515 for AR-RU harm and 0.598 for ID-VI harmless.
  • Gemma sharing patterns: Gemma shows strongest harm convergence in middle layers and reverses the usual late-layer ordering, with late harm sharing exceeding harmless sharing.This contrasts with Llama and Qwen, where late harmless sharing is higher.
Loading 2608.29936v1…