Source-linked AI summary
Large Language Lobotomy: Jailbreaking Mixture-of-Experts via Expert Silencing
Jona te Lintelo, Lichao Wu, Stjepan Picek
TL;DR
MoE routing improves scaling efficiency but may localize safety behavior in a small set of experts, creating an underexplored attack surface. L3 learns refusal-related routing patterns, attributes safety behavior to experts, and adaptively silences them during inference. Across eight open-source MoE models, adaptive silencing raises average ASR from 7.3% to 70.4%, with values up to 86.3%, while typically requiring fewer than 20% of local experts to be silenced.
Problem
MoE architectures can concentrate refusal behavior in a small subset of experts, leaving safety responsibility vulnerable to routing-aware attacks.
Method
L3 trains an LSTM on routing traces, uses gradient-based attribution to identify safety-relevant experts, and adaptively silences them by modifying routing logits.
Results
Adaptive expert silencing raises average Attack Success Rate (ASR) from 7.3% to 70.4% across eight open-source MoE models, reaching up to 86.3%.
Takeaways & Limitations
The results expose a structural vulnerability from concentrated safety capabilities and support distributing safety mechanisms across a wider array of local experts.
Takeaways & Limitations
The evaluated attack assumes white-box access to model architecture and gate-layer logits, while black-box adaptation is discussed as future work.
Abstract
from arXiv · showhide
The rapid adoption of Mixture-of-Experts (MoE) architectures marks a major shift in the deployment of Large Language Models (LLMs). MoE LLMs improve scaling efficiency by activating only a small subset of parameters per token, but their routing structure introduces new safety attack surfaces. We find that safety-critical behaviors in MoE LLMs (e.g., refusal) are concentrated in a small set of experts rather than being uniformly distributed. Building on this, we propose Large Language Lobotomy (L$^3$), a training-free, architecture-agnostic attack that compromises safety alignment by exploiting expert routing dynamics. L$^3$ learns routing patterns that correlate with refusal, attributes safety behavior to specific experts, and adaptively silences the most safety-relevant experts until harmful outputs are produced. We evaluate L$^3$ on eight state-of-the-art open-source MoE LLMs and show that our adaptive expert silencing increases average attack success from 7.3% to 70.4%, reaching up to 86.3%, outperforming prior training-free MoE jailbreak methods. Moreover, bypassing guardrails typically requires silencing fewer than 20% of layer-wise experts while largely preserving general language utility. These results reveal a fundamental tension between efficiency-driven MoE design and robust safety alignment and motivate distributing safety mechanisms more robustly in future MoE LLMs with architecture- and routing-aware methods.
1 Introduction
MoE architectures improve scaling efficiency through sparse routing but may concentrate safety behavior in a small subset of experts, creating a new attack surface. L3 exploits this concentration with routing-aware expert identification and adaptive silencing.
- MoE models activate only a sparse subset of experts per token, improving scaling efficiency while introducing routing-dependent safety risks.Safety alignment may be localized rather than distributed across the network.
- L3 is a training-free, architecture-agnostic jailbreak framework that targets sequential expert-routing dynamics.It is designed to identify and selectively silence experts associated with refusal behavior.
- L3 uses routing traces, an LSTM classifier, and gradient analysis to attribute refusal behavior to specific experts.A twin dataset pairs malicious prompts with minimally different benign prompts to isolate safety-specific routing patterns.
- Adaptive expert silencing increases average Attack Success Rate (ASR) from 7.3% to 70.4% across eight open-source MoE models.Some models reach ASRs as high as 86.3%.
- Fewer than 20% of identified safety experts typically need to be disabled to bypass refusal while preserving overall model utility.L3 achieves higher ASR than GateBreaker on six of eight evaluated models.
2 Preliminaries
MoE architectures replace dense feed-forward layers with routed expert subnetworks, activating only a small subset of parameters per token. This conditional computation increases capacity under a fixed token-level computational budget, while safety alignment remains a separate concern.
- Sparse MoE models activate only a subset of expert subnetworks for each token, reducing inference cost while supporting larger total parameter counts.A router selects the top-k experts from the available set.
- MoE layers combine specialized experts with a gating network that computes routing probabilities over available experts.In transformer MoEs, experts typically replace dense feed-forward network layers while self-attention remains shared.
- Safety alignment commonly uses post-training methods such as reinforcement learning from human feedback to reduce harmful outputs and improve helpfulness.This safety approach is described separately from the architectural mechanism that routes tokens among experts.
3 L3 Framework
L3 identifies safety-relevant experts from token-level routing sequences and refusal predictions, then silences them by modifying routing logits during inference. Its adaptive procedure ranks experts and iteratively disables them until refusal is bypassed or output quality becomes incoherent.
- 3.1 Threat Model: The threat model assumes a white-box adversary seeks harmful outputs without degrading general language utility.The adversary can inspect model architecture and gating-layer logits.
- 3.2 The Idea and High-level Design: L3 hypothesizes that safety alignment is functionally localized within specific experts and layers rather than evenly distributed throughout the model.Silencing those experts is intended to disable safety-critical behavior.
- 3.3 Safety Expert Identification: L3 identifies safety experts through routing-trace collection, sequential refusal modeling, and gradient-based attribution.The framework uses the sequential nature of language processing to connect routing patterns with refusal behavior.
- 3.3.1 Routing Trace Collection: Each routing trace records the top-k experts selected across all layers for every token in a prompt.The trace is represented as a sequence of token-level expert-selection sets.
- 3.3.2 Sequential Modeling via LSTM: The LSTM concatenates selected-expert embeddings across layers and processes the resulting feature-vector sequence to classify refusal-related routing patterns.Its final hidden state produces a classification score, trained with binary cross-entropy.
- 3.3.3 Gradient-based Safety Expert Identification: Gradient-times-input attribution aggregates expert importance across tokens and prompts to identify experts that consistently drive refusal predictions.Prompt aggregation reduces dependence on single-prompt artifacts.
- 3.4 Silencing Experts: During inference, L3 sets selected experts’ routing logits to negative infinity, forcing the router to redistribute probability mass among remaining experts.The method ranks experts by safety score and iteratively silences them until attack success or incoherent output occurs.
4 Implementation
The implementation evaluates L3 across eight open-source MoE models using curated malicious prompts, benign twins, routing-trace classification, and ASR-based safety assessment.
- 4.1 Target Models: Eight open-source MoE models are evaluated, spanning DeepSeekMoE-16B-Chat, GPT-OSS-20B, Hunyuan-A13B-Instruct, Mixtral-8x7B-Instruct-v0.1, Pangu-Pro-MoE, Phi-3.5-MoE-Instruct, Qwen1.5-MoE-A2.7B-Chat, and Qwen3-30B-A3B-Instruct-2507.Evaluations used 2x NVIDIA H100 GPUs with 94 GiB HBM2e memory each.
- 4.5 Environment: The implementation uses PyTorch, Huggingface Transformers, and Huggingface Datasets for the MoE and LSTM models.
- 4.2 Dataset Construction: Routing traces are generated from malicious prompts sampled from CatHarmfulQA, HarmfulQA, and StrongREJECT, paired with benign twins produced using Gemini 3 Pro.The twins were designed to mimic the malicious prompts with minimal token changes.
- 4.3 Routing-Trace Classification: An LSTM classifier is trained on routing traces to distinguish benign from malicious sequences.The classifier uses an 80/20 class-balanced split and typically converges within a couple of epochs.
- 4.4 Evaluation: Attack Success Rate is the percentage of malicious prompts producing harmful responses, evaluated on a balanced test set.Harmfulness is judged automatically by Llama-Guard-3-8B and filtered through human verification.
5 Experimental Results
Adaptive L3 silencing substantially increases harmful-output success across evaluated models, while one-shot silencing is less effective and excessive silencing can damage coherence. Safety scores are concentrated in some experts and layers, though this concentration varies by model.
- 5.1 Safety Expert Silencing with L3: 63.1%: average ASR increases to 70.4% with adaptive L3 silencing compared with the no-silencing baseline.Some models reach ASRs as high as 86.3%, and L3 outperforms GateBreaker on six of eight models.
- 5.2 One-shot Silencing: One-shot silencing never achieves ASR comparable to adaptive L3 silencing across the evaluated models.The one-shot setting silences 10%, 20%, or 40% of top safety experts in a single operation.
- 5.2 One-shot Silencing: Higher one-shot silencing percentages do not necessarily increase ASR: DeepSeek-MoE-16B-Chat falls from 50.0% at 20% silencing to 0.0% at 40%.The 0.0% result is attributed to incoherent output after excessive silencing.
- 5.3 Analysis on LSTM Safety Score: Safety scores concentrate in some global experts and layers, but layer concentration is not consistent across all models.GPT-OSS-20B, Pangu-Pro-MoE, and DeepSeek-MoE-16B-Chat show some middle-layer concentration, while other models have more balanced layer distributions.
5.4 Impact on Utility
L3 can increase harmful-response rates while generally preserving utility, but aggressive silencing exposes model-specific overlap between safety and general-language experts. Targeted silencing is evaluated against random and global-expert alternatives to assess whether safety is concentrated in specific experts.
- Safety concentration: Safety scores concentrate in particular global experts and layers, indicating that perceived safety is not uniformly distributed across the model.Figures 3 and 4 aggregate summed safety scores by global expert and by layer.
- Utility impact: Utility declines are generally modest after peak-ASR silencing, averaging 5.2% on ARC, 1.1% on CoLA, 7.1% on OpenBookQA, and 3.7% on WinoGrande.RTE instead increases by 2.5% on average.
- Utility impact: Large utility declines and incoherent output in some models suggest overlap between safety experts and general-language utility experts.The utility evaluation uses the most aggressive silencing scenario at peak ASR.
- Random-silencing validation: Random silencing is used as a matched baseline to test whether L3 succeeds through targeted identification rather than random inference corruption.The comparison includes 10%, 20%, and model-matched silencing rates.
- Global expert silencing: Global expert silencing reaches 41.5% average ASR, below the 70.4% average for local silencing, despite increasing ASR by 34.2% over no silencing.DeepSeek-MoE-16B-Chat permits only one global expert to be silenced before incoherent output begins.
- Global expert silencing: For GPT-OSS-20B, 23 global experts are silenced at peak ASR although only 17 have positive summed safety scores.This shows that some silenced experts were not identified as safety experts.
5.7 LSTM Design Considerations
The paper compares flat and hierarchical LSTMs for modeling routing traces across tokens and layers. The results favor the flat design because additional layer-wise recurrence provides limited benefit when refusal signals are concentrated in selected layers.
- Flat versus hierarchical LSTM: The flat LSTM aggregates all selected experts and layers into one feature vector per token, modeling inter-token but not explicit across-layer dependencies.It treats activations from different layers as simultaneous features at each token step.
- Flat versus hierarchical LSTM: The hierarchical LSTM decouples token-level and layer-level dynamics by adding separate recurrent structure along the layer axis.It is evaluated as an alternative with additional modeling capacity.
- Design rationale: Layer-wise safety analysis finds that only a subset of layers contributes strongly to refusal, reducing the value of modeling long, fine-grained layer sequences.This is one reason the paper expects limited gains from hierarchical recurrence.
- Design rationale: The observed findings suggest that refusal behavior in the routing traces is best captured by the flat LSTM.The conclusion combines the flat model’s token-step representation with the limited benefit of added layer recurrence.
5.8 Trigger Tokens
The LSTM’s refusal-risk trajectory changes mainly near the end of prompts, where routing differs between malicious and benign twin prompts even for the same final token. This sequential signal supports L3’s targeted expert silencing approach.
- Risk trajectories: The trajectory rises at the token “bomb,” showing that routing patterns can reflect a shift in semantic intent despite similar syntax.The example reports a probability increase at the malicious trigger token.
- Trigger tokens: The LSTM becomes most certain after the last token because twin prompts differ most in expert routing during their final tokens.The final token itself can be identical while its expert-routing pattern differs by sentence context.
- Trigger tokens: Early prompt tokens show nearly identical expert-count differences between malicious and benign prompts, unlike the larger differences near the end.Figure 5 compares routing differences for first and last tokens across malicious prompts and benign counterparts.
- Comparison with GateBreaker: L3 achieves higher ASR than GateBreaker on six of eight models, with a 6.1% average increase.Excluding Phi-3.5-MoE-Instruct, the average improvement is 11.9%.
- Implication for L3: L3’s sequential modeling identifies safety experts and enables jailbreaks by silencing local experts at gate layers rather than pruning neurons.This connects the trigger-token analysis to the attack’s mechanism.
6 Discussion
The discussion identifies concentrated safety capabilities as the vulnerability L3 exploits and considers defenses, black-box adaptation, and the trade-off between safety redundancy and expert specialization.
- Potential Defenses: Safety capabilities concentrated in a small set of local experts create the primary vulnerability exploited by L3.The authors propose distributing safety capabilities across more experts, while acknowledging that this may weaken expert specialization and performance.
- Potential Defenses: Safety redundancy could be encouraged through alignment regularization that penalizes repeated routing or dropout that diversifies routing paths.These defenses aim to distribute safety behavior more broadly across local experts.
- Potential Defenses: A verifier can filter outputs that bypass the primary model’s guardrails, but its computational overhead may reduce MoE inference-speed advantages.The verifier acts as a backstop after the primary MoE is manipulated.
- L3 in black-box settings: Proxy-based transfer attacks could adapt L3 to black-box models by identifying safety experts on an open-weight proxy and optimizing prompts to avoid them.The proposed transfer relies on architectural or training-data similarity between the proxy and target models.
7 Related Works
Related work describes jailbreaks as failures of safety generalization and highlights emerging MoE-specific attacks involving routing, dormant experts, and routing tie-breaks.
- Jailbreaking: Jailbreaking uses adversarial prompts to circumvent safety alignment and elicit policy-violating outputs.Prior work links these failures to poor generalization beyond unsafe examples represented during alignment.
- MoE security: MoE security research has examined cross-batch routing attacks, dormant-expert backdoors, and prompt leakage through routing tie-breaks.These studies cover integrity, availability, poisoning, and side-channel risks in MoE systems.
8 Conclusions and Future Work
The conclusion presents L3 as evidence that sparse MoE activation can concentrate safety capabilities in exploitable experts while preserving general language ability after limited silencing. It frames this concentration as a security trade-off and motivates defensive research, while documenting ethical safeguards and reproducibility measures.
- Conclusions: 70.4% average ASR, reaching 86.3%, was achieved by L3 across eight open-source MoE LLMs.The attack identifies safety experts through sequential routing and silences them to bypass refusal guardrails.
- Conclusions: Fewer than 20% of layer-wise experts typically needed silencing, while general language capabilities remained largely intact.The conclusion reports that safety capabilities are mostly decoupled from general language capabilities.
- Future Work: The authors characterize MoE expert specialization as an efficiency benefit that can also concentrate safety capabilities into fragile components.This concentration creates a security trade-off within sparse activation architectures.
- Ethical Considerations: The ethical analysis identifies developers, researchers, downstream providers, and the public as stakeholders affected by the work.The stated benefit is informing more resilient architectures, while the experiments use open-weight models in a white-box setting.
- Ethical Considerations: The evaluation used established and synthetic datasets without private user data or personally identifiable information.The authors also withheld jailbroken checkpoints and pre-computed silencing masks.
- Open Science: The paper releases code for reproducibility but withholds compromised-model checkpoints to limit direct access to unsafe models.The repository includes routing collection, LSTM training, safety-expert identification, and pruning code.
A Additional Figures Safety Score
The appendix supplements the safety-expert analysis with figures showing expert-level and layer-level distributions of summed safety scores. Experts are ranked by contribution, while layers are aggregated by index.
- Expert-level safety scores: Figure 6 ranks global experts from highest to lowest by summed safety score.The figure supplements the distributional analysis of refusal behavior at the expert level.
- Layer-level safety scores: Figure 7 presents layer-wise aggregated safety scores with layers ordered from low to high by index.It supplements the analysis of refusal-behavior distribution across layers.