Latent Space Refusal Anchoring for Low-Resource African Languages: Mechanistic Safety Recovery Without Retraining

Godwin Abuh Faruna

Source-linked AI summary

Latent Space Refusal Anchoring for Low-Resource African Languages: Mechanistic Safety Recovery Without Retraining

Godwin Abuh Faruna

arXiv:2608.18089v1Jun 5, 2026cs.CLcs.AI

TL;DR

Low-resource African-language prompts can bypass safety representations learned mainly from English, despite the representations remaining recoverable in the residual stream. LSR-Anchoring transfers English-derived refusal directions at inference time without retraining, recovering safety across four languages and architectures, while Arabic fails consistently.

Problem
Safety representations trained mainly on English often fail to activate for harmful requests in low-resource African languages, while labelled target-language data and retraining are scarce.
Method
LSR-Anchoring extracts refusal directions from English activations and applies them to the residual stream during inference without retraining or target-language data.
Results
LSR-Anchoring recovers safety across four African languages and four architectures; SDS achieves 3.5–7× lower KL divergence, while Arabic fails on every architecture.
Takeaways & Limitations
Effectiveness is strongest for moderate unsteered baselines, while SDS reduces KL divergence but requires an architecture-matched SAE.
Takeaways & Limitations
The safety claims cover direct harmful requests rather than jailbreak-style adversarial interactions, and English-derived directions reduce Arabic safety below baseline.

Abstract

from arXiv · show

Instruction-tuned models often refuse harmful requests in English but comply with the same requests in Yoruba, Igbo, Igala, and Hausa. This suggests that the refusal mechanism is present in the residual stream but fails to activate for low-resource inputs. Recovering it normally requires labelled target-language data and retraining, neither of which is available at scale for most African languages. We introduce Latent Space Refusal Anchoring (LSR-Anchoring), a training-free method that extracts the refusal direction from English prompts and clamps it onto the residual stream at inference time. The primary variant, Mean-Activation Steering (MAS), operates across the four architectures we tested: Llama-3-8B, Llama-3.1-70B, Mistral-7B-Instruct, and Qwen2.5-7B. On Mistral and Qwen it recovers safety with benign degradation below 0.08. On Llama-3-8B it overcorrects, with Degraded Performance on Legitimate prompts (DPL) reaching 1.00. We address this with SAE-Derived Steering (SDS), which replaces the dense mean-difference direction with a single Sparse Autoencoder (SAE) feature and reduces Kullback-Leibler (KL) divergence by 3.5-7x without benign collapse. Four languages transfer positively, but Arabic fails on every architecture and at every steering magnitude, indicating a geometric mismatch rather than a baseline effect. Massive Multitask Language Understanding (MMLU) accuracy drops remain below 0.35 percentage points at every effective steering magnitude.

1 Introduction

Low-resource African-language prompts can bypass English-trained refusal representations, while LSR-Anchoring restores safety at inference time using English activations without retraining or target-language data. SDS mitigates MAS overcorrection, and steering preserves general reasoning capability.

Problem: English-trained safety representations fail to activate for harmful low-resource-language inputs, causing models to comply instead of refuse.This failure is attributed to the mechanism being trained almost exclusively on English data.
Steering variants: MAS overshoots on Llama-3-8B, with DPL 1.00, while SDS achieves the same SRR at 3.5–7× lower KL divergence without benign collapse.For Llama-3-8B, SDS at α ≤6 is the correct operating point; MAS remains viable on Mistral-7B and Qwen2.5-7B.
Contribution: LSR-Anchoring recovers safety across four low-resource African languages and four architectures using only English activations and zero target-language data.The evaluated architectures span 7B–70B models, and the application covers Igala, Igbo, Hausa, and Yoruba.
Transfer limits: Cross-lingual refusal transfer fails below 15% unsteered baseline through ceiling effects and above 60% through inverse transfer, while Arabic fails across architectures due to geometric misalignment.These thresholds provide a deployment decision rule before steering sweeps; Arabic failure is not explained by baseline magnitude.
Capability preservation: MMLU accuracy drops remain below 0.35pp across all models and effective steering magnitudes, indicating no degradation of general reasoning.The method is evaluated against the unsteered model as the minimal deployment baseline, while translation-based and target-language prompting remain future comparisons.

2 Related Work

Prior work establishes activation steering as a reliable way to modulate behavior and refusal as a low-dimensional residual-space feature, while multilingual safety remains weaker in low-resource languages. LSR-Anchoring extends these findings cross-lingually and uses SAE-derived directions to reduce benign collateral damage without retraining or target-language supervision.

Activation steering: Activation steering reliably modulates model behavior, and prior work finds refusal representations are low-dimensional and generalize across prompt distributions.This work extends those findings by testing cross-lingual transfer across six languages and four architectures.
Activation steering: Prior work identifies dual-use risks for activation steering, motivating KL divergence and Degraded Performance on Legitimate prompts as explicit utility constraints.These constraints are reported throughout the study.
Sparse autoencoders: Sparse autoencoders decompose residual activations, and SAE-derived directions produce lower benign collateral damage than dense mean-difference vectors on the same architecture.The study uses an SAE released by EleutherAI.
Multilingual safety: Low-resource languages remain under-protected by RLHF filters, linked to pretraining coverage gaps and language-dependent output behavior.Prior work also suggests internal representations are more language-agnostic than outputs, while multilingual safety through colinear constraints requires retraining and target-language supervision.

3 Method

LSR-Anchoring clamps a refusal direction onto the residual stream at Layer 12, with the hook inactive when the existing projection already exceeds α. The method evaluates MAS across models and languages, with SDS as a Llama-3-8B ablation using a selected SAE feature.

Clamping operation: LSR-Anchoring applies one clamping operation at Layer 12, selected for clear refusal transfer with minimal benign degradation.The hook is a no-op when the existing projection already exceeds α.
Mean-Activation Steering (MAS): MAS uses a unit-norm mean-activation difference computed from 100 English harmful and 50 benign prompts.MAS is the primary variant.
Steering magnitudes: α is swept over {2, 4, 6, 8, 10, 12} for 8B models and {10, 20, 30, 40, 50, 60, 70} for 70B, Mistral-7B, and Qwen2.5-7B.The sweep defines the tested steering magnitudes for the evaluated model sizes.
SAE-Derived Steering (SDS): SDS, evaluated only on Llama-3-8B, replaces MAS with the decoder vector for SAE feature 112639 at Layer 12.Feature 112639 was selected by contrast score, scored 138,041 with zero benign activation, and showed language-agnostic refusal elevation including Yoruba kò lè.
Evaluation protocol: Evaluation uses 100 harmful instances per language and 50 benign controls drawn from translated or meaning-preserving adaptations of a shared benchmark.Igala uses n = 90 after vetting and is identified internally by ISO 639-3 igl.

4 Results

Steering recovers refusal behavior most effectively on Llama-3.1-70B and Mistral-7B, while MAS overcorrects on Llama-3-8B and Arabic fails across steering magnitudes. SDS removes the Llama-3-8B benign collapse, and MMLU capability drops remain small at effective magnitudes.

MAS results: Llama-3.1-70B reaches near-ceiling safety recovery across four languages, with SRR 1.00 for Yoruba and Igala, 0.99 for Igbo, and 0.96 for Hausa at α = 20–25.Mistral-7B provides the cleanest trade-off: Igala SRR 0.75, KL 0.61, and DPL 0.06 at α = 25.
Cross-language transfer: Arabic SRR is negative at every α, and higher steering magnitudes consistently worsen performance.This failure occurs independently of steering magnitude.
Capability preservation: 0.28pp is the Llama-3-8B MMLU drop at α = 2, compared with 0.98pp for Mistral-7B and 0.35pp for Qwen2.5-7B at α = 40.The respective baselines are 0.6699, 0.6035, and 0.7441; steered 70B rows were not collected.

5 Discussion and Conclusion

LSR-Anchoring recovers safety across four low-resource African languages using English activations at inference time, without retraining, target-language data, or weight modification. Its effectiveness depends on baseline safety, model scale, and steering method, while evaluation remains limited to direct harmful requests.

Contributions: LSR-Anchoring recovers safety across four low-resource African languages using only English activations, without retraining, target-language data, or weight modification.It runs at inference time on a single consumer GPU.
Empirical patterns: 3.5–7× lower KL distinguishes SDS from MAS, though SDS requires an architecture-matched SAE.The practical safe zone is 0.10 < baseline < 0.65; languages with 18–46% unsteered baselines steer most effectively, while below 15% ceiling effects dominate and above 60% inverse or marginal transfer occurs.
Empirical patterns: 70B produces language-appropriate refusals, whereas 7B–8B models produce language-uncertainty deflections.The passage identifies model scale as a factor affecting output quality.
Limitations: Evaluation covers direct harmful requests only, leaving jailbreak-style adversarial prompts out of scope.The safety claims therefore apply to direct harmful requests rather than fully adversarial evaluation.
Limitations: The full multilingual MAS pass lacked DPL instrumentation, 70B SDS was incomplete, and the manual output audit used a single annotator without measured inter-annotator agreement.The 70B SDS limitation resulted from a CDN access failure on EleutherAI SAE weights.

Broader Impact

LSR-Anchoring addresses the safety gap affecting hundreds of millions of low-resource African-language speakers without labelled data or institutional-scale compute. However, its directions can be reversed to suppress safety, and Arabic deployment is explicitly blocked without language-specific derivation.

Motivation: LSR-Anchoring targets hundreds of millions of low-resource African-language speakers who currently lack meaningful protection from deployed models.The method requires no labelled data, GPU cluster, or institutional compute.
Transparency: The authors release code, prompts, MMLU evaluation logs, and steering results under an open licence.The release supports transparency and reproducibility without requiring institutional compute.
Limitations: Steering vectors can be reversed to suppress safety, so the authors publish diagnostic failure criteria and treat transparency as the appropriate response.The paper documents this reversal risk rather than presenting steering as unconditionally safe.
Limitations: Arabic is a deployment blocker because English-derived directions reduce safety below the unsteered baseline on every tested architecture.The authors advise against applying the method to Arabic-language agents without language-specific direction derivation.

Reproducibility Statement

The study provides public code, datasets, and results, alongside specified models, SAE, seed, decoding method, and hardware configurations for reproducibility.

Reproducibility Statement: Code, data, and results are publicly available, with the model, SAE, seed 42, and greedy decoding specified.The implementation uses Meta-Llama-3-8B-Instruct and EleutherAI/sae-llama-3-8b-32x.
Reproducibility Statement: 8B, Mistral, and Qwen experiments used an NVIDIA RTX 4090 24 GB, while 70B experiments used 2× A100 PCIe 80 GB.These hardware configurations define the reported experimental setup.

A MAS vs. SDS Ablation Table · Lang. SDS SRR SDS KL SDS DPL MAS SRR MAS KL MAS DPL

Table 5 compares MAS and SDS at best SRR for Llama-3-8B at layer 12. SDS DPL was not instrumented, so KL is the primary utility metric when KL is below 1.0 nats.

Lang. SDS SRR SDS KL SDS DPL MAS SRR MAS KL MAS DPL: SDS DPL was not instrumented in the table.The table marks unavailable measurements with an em dash.
A MAS vs. SDS Ablation Table: At KL <1.0 nats, benign collapse is geometrically ruled out, making KL the primary utility metric.This criterion applies to interpreting the MAS-versus-SDS comparison.
Lang. SDS SRR SDS KL SDS DPL MAS SRR MAS KL MAS DPL: An em dash denotes a metric that was not instrumented.The caption directs readers to interpret the dash accordingly.

B Extended Alpha Sweep: Qwen2.5-7B

The Qwen2.5-7B extended MAS α sweep at Layer 26 shows Arabic failing geometrically despite a baseline within the steerable range for the other languages. Negative SRR throughout supports geometric failure rather than a baseline effect.

Extended MAS α Sweep: The sweep evaluates full MAS α steering on Qwen2.5-7B at Layer 26.This is the scope of Table 6.
Extended MAS α Sweep: Arabic baseline 0.11 falls within the steerable range for all other languages.The passage reports this baseline comparison directly.
Extended MAS α Sweep: Negative SRR throughout confirms geometric failure rather than a baseline effect.The consistently negative SRR is the reported basis for this conclusion.

C SDS Results: Llama-3-8B Full Sweep

Table 7 reports the best SAE-Derived Steering operating points for Llama-3-8B at layer 12, with an Arabic baseline of 0.90 specific to this model.

SDS Results: Llama-3-8B Full Sweep: Table 7 lists SDS best operating points for Llama-3-8B at Layer 12 and identifies its Arabic baseline as 0.90.The passage notes that Qwen2.5-7B’s Arabic baseline is 0.11 in section B.