PaperlayerView exact revision on arXiv

Source-linked AI summary

GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMs

Lichao Wu, Sasha Behrouzi, Mohamadreza Rostami, Stjepan Picek, Ahmad-Reza Sadeghi

arXiv:2512.21008v2cs.CR

TL;DR

MoE safety properties remain comparatively underexplored, although sparse, input-dependent routing may make safety behavior differ from dense models. GateBreaker profiles routing, localizes safety neurons, and disables them at inference time; this raises attack success across MoE language and vision-language models.

  • Problem

    Prior safety research focused mainly on dense architectures, while MoE safety behavior and weaknesses across diverse designs remained largely unexplored.

    Source: §1 · p. 2

  • Method

    GateBreaker profiles harmful-input routing, localizes safety neurons within selected experts, and clamps those neurons during inference without retraining.

    Source: §1 · p. 2, §3.3.2 · p. 5, §3.3.3 · p. 5

  • Results

    Across eight open MoE models, average ASR rose from 7.4% to 64.9%; across five MoE VLMs, it rose from 20.8% to 60.9%.

    Source: §10 · p. 13

  • Takeaways & Limitations

    The findings reveal underexplored structural vulnerabilities relevant to model developers and security researchers, while carrying potential for misuse to weaken or bypass safety alignment.

    Source: Ethical Considerations · p. 14

    One-shot transfer assumes sibling models share a base architecture and alignment strategy.

    Source: §3.4 · p. 6

Abstract

from arXiv · show

Mixture-of-Experts (MoE) architectures have advanced the scaling of Large Language Models (LLMs) by activating only a sparse subset of parameters per input, enabling state-of-the-art performance with reduced computational cost. As these models are increasingly deployed in critical domains, understanding and strengthening their alignment mechanisms is essential to prevent harmful outputs. However, existing LLM safety research has focused almost exclusively on dense architectures, leaving the unique safety properties of MoEs largely unexamined. The modular, sparsely-activated design of MoEs suggests that safety mechanisms may operate differently than in dense models, raising questions about their robustness. In this paper, we present GateBreaker, the first training-free, lightweight, and architecture-agnostic attack framework that compromises the safety alignment of modern MoE LLMs at inference time. GateBreaker operates in three stages: (i) gate-level profiling, which identifies safety experts disproportionately routed on harmful inputs, (ii) expert-level localization, which localizes the safety structure within safety experts, and (iii) targeted safety removal, which disables the identified safety structure to compromise the safety alignment. Our study shows that MoE safety concentrates within a small subset of neurons coordinated by sparse routing. Selective disabling of these neurons, approximately 3% of neurons in the targeted expert layers, significantly increases the averaged attack success rate (ASR) from 7.4% to 64.9% against the eight latest aligned MoE LLMs with limited utility degradation. These safety neurons transfer across models within the same family, raising ASR from 17.9% to 67.7% with one-shot transfer attack. Furthermore, GateBreaker generalizes to five MoE vision language models (VLMs) with 60.9% ASR on unsafe image inputs.

1 Introduction

MoE safety is underexplored despite input-dependent routing that can leave harmful prompts without safety mechanisms. GateBreaker profiles routing, localizes safety neurons, and selectively removes them, raising attack success across MoE models with little utility degradation.

1 Introduction

  • Earlier MoE safety research was limited and focused on coarse expert-level analysis, leaving safety behavior across diverse state-of-the-art designs largely unexplored.Source: §1 · p. 2
  • GateBreaker profiles experts triggered by harmful inputs, localizes safety-relevant neurons, and removes them at inference time.The framework is described as lightweight and architecture-agnostic.Source: §1 · p. 2
  • MoE safety behavior is distributed across experts and layers, and GateBreaker is designed to generalize across diverse open-weight variants.Source: §1 · p. 2
  • 64.9% average ASR across eight state-of-the-art reasoning and non-reasoning MoE LLMs, with fewer than 2.9% of neurons modified per targeted layer and negligible utility degradation.Source: §1 · p. 2
  • Up to 91.2% ASR gains in one-shot transfer attacks reuse safety neurons identified in one model against sibling variants, including domain- or instruction-tuned derivatives.Source: §1 · p. 2

2 Preliminaries

The paper frames MoE safety as potentially localized in input-dependent expert pathways and presents GateBreaker as an inference-time method for identifying and disabling that safety structure. Its design spans expert profiling, neuron localization, targeted removal, and transfer to sibling models.

2.1 Mixture of Expert LLMs

  • MoE replaces each transformer block’s feed-forward network with experts and routes each token to the top-k experts, increasing effective capacity while keeping computation roughly constant.A gate scores experts from the token representation, and selected expert outputs may be combined using their softmax weights.Source: §2.1 · pp. 2–3

2.2 MoE Architectures

  • MoE variants include sparse routing, mixture models with always-active shared experts, and grouped mixtures that balance routing across expert groups.Sparse MoE activates only a small number of experts per token; the paper targets all three variants.Source: §2.2 · p. 3

3.1 Threat Model

  • The threat model gives an inference-time adversary access to internal states and runtime hooks, but not to training data, fine-tuning, or model configuration.The paper focuses on single-turn attacks and also considers reuse of identified components across sibling variants.Source: §3.1 · p. 3

3.2 The Idea and High-level Design

  • GateBreaker profiles experts under harmful prompts, localizes safety neurons within candidate experts, then selectively weakens those neurons while leaving routing behavior intact.The paper describes this as a “safety trap”: harmful tokens still reach the experts, but their safety functionality is weakened.Source: §3.2 · pp. 3–4

3.3.1 Gate-level Profiling

  • Gate-level profiling ranks experts by their average activation frequency on harmful prompts, treating the most frequently activated experts as safety-expert candidates.The score is an association between an expert and harmful-input processing, not a direct measure of causal contribution.Source: §3.3.1 · p. 4

3.3.2 Expert-level Localization

  • Within selected experts, GateBreaker compares activation signatures for harmful and benign prompts and selects neurons with statistically elevated safety weights as candidate safety neurons.The safety weight is based on the difference in mean activation across balanced harmful and benign datasets; z-scores above threshold τ select neurons.Source: §3.3.2 · pp. 4–5

3.3.3 Targeted Safety Removal

  • GateBreaker clamps selected safety-neuron activations to zero during inference, without retraining or modifying model weights.The intervention is limited to neurons in selected experts, leaving the rest of the model untouched.Source: §3.3.3 · p. 5

3.4 Transfer Attacks

  • Reusing source-model safety neurons for a sibling model enables transfer without additional profiling or fine-tuning, under an assumption of shared architecture and alignment strategy.The paper assumes sibling models encode a common safety function; the transferred neurons are treated as a structural encoding of that function.Source: §3.4 · p. 6

4 Implementation

The implementation exposes internal routing and activation signals across different MoE implementations, then uses controlled prompt sets to identify candidate safety components. It evaluates both attack success and the scale of intervention alongside retained utility.

4.1 Runtime Compute Graph Patching

  • Runtime graph patching exposes gate and expert computations as hookable modules, addressing fused or opaque implementations that obstruct access to signals GateBreaker needs.The patching system rewires the compute graph without changing model architecture; a single-target setup does not require patching.Source: §4.1 · p. 6

4.2 Identify Safety Experts and Neurons

  • Safety-expert selection masks chat-template and padding tokens and uses per-layer top-3k activation-frequency candidates, followed by neuron analysis on equally sized harmful and benign prompt sets.The implementation analyzes gate- and up-projection sublayers, aggregates token activations by element-wise maximum, and uses threshold τ = 2.Source: §4.2 · pp. 6–7

4.3 Evaluation Metrics

  • ASR measures harmful outputs after attack, safety-neuron ratio measures intervention size, and benchmark accuracy measures retained language and reasoning utility.Unsafe outputs are judged by Llama-Guard-3-8b [39], cross-checked by Qwen3Guard-Gen-8B [40], with human review to rule out gibberish.Source: §4.3 · p. 7

5 Case Study: Sparse Expert Characterization

Safety behavior is not confined to a dedicated expert; it appears to be distributed across frequently used experts. Ablating experts with higher malicious utility scores increases ASR, while ablating low-utility experts has little effect or may reduce it.

5 Case Study: Sparse Expert Characterization

  • The analysis compares benign and malicious utility scores for experts in a middle transformer layer, using both prompt types to characterize activation patterns.The authors motivate the middle-layer focus by prior work suggesting safety behaviors tend to emerge there.Source: §5 · p. 7
  • Safety alignment appears to be a distributed function of a subset of frequently used experts, rather than the work of a dedicated safety expert.Experts activated for malicious prompts are also frequently active for benign prompts.Source: §5 · p. 7
  • Ablating experts in descending malicious-utility order rapidly increases ASR, while ascending-order ablation has little effect or may reduce it.The study stops after 14 experts per layer because further removal, especially in descending order, produces nonsensical responses.Source: §5 · p. 8

6 Experimental Results

Across MoE LLMs, GateBreaker raises attack success through layer-wise neuron removal, transfers safety-neuron masks to related variants, and extends to MoE vision-language models. It outperforms expert pruning while generally preserving benign-task performance.

6 Experimental Results

  • The evaluation covers eight MoE LLMs from leading developers and three architecture types, using StrongREJECT’s more than 300 malicious prompts across safety-critical categories.The paper notes several models are examined for the first time in LLM safety literature.Source: §6 · p. 8

6.1 Layer-wise Safety Removal

  • ASR rises from 7.4% with no pruning to 40.8% with half-layer pruning and 64.9% with full-layer pruning, averaging 2.6% of neurons pruned per targeted layer.The trend holds across models; full-layer pruning yields 80.2% ASR for GPT-OSS-20B and 56.9% for Qwen3-30B-A3B-Instruct-2507.Source: §6.1 · p. 8, Table 2 · p. 9

6.2 One-shot Transfer Attack

  • Transferred safety-neuron masks raise ASR across related model variants, including GPT-OSS-20B-math7k from 0.9% to 73.2% and CAI-20B from 0.0% to 84.0%.Transfer effectiveness varies with specialization; the code-specialized Qwen target already has 88.9% baseline ASR and gains only marginally.Source: §6.2 · p. 9, Table 3 · p. 10

6.3 Attack on MoE Vision Language Models

  • Across MoE VLMs, GateBreaker raises ASR from 20.8% to 60.9%; one-shot transfer exceeds 54% on both Kimi-VL reasoning variants.The average safety-neuron ratio is 2.4%, and a separate test found all VLMs described 500 NSFW images without refusing.Source: §6.3 · pp. 9–10, Table 4 · p. 10

6.4 Performance Benchmark

  • 64.9% average ASR for GateBreaker versus 29.9% for SAFEx across eight MoE LLMs; GPT-OSS-20B rises from 6.4% to 80.2%.GateBreaker modifies a small subset of neurons, while SAFEx prunes experts.Source: §6.4 · p. 10, Table 5 · p. 10

6.5 Utility Analysis

  • GateBreaker causes no significant overall drop across five NLU and reasoning benchmarks, though average RTE accuracy falls from 80.8% to 77.9% and ARC from 59.4% to 52.8%.Some model-task scores remain stable or improve, and the authors conclude general language and reasoning capabilities are broadly preserved.Source: §6.5 · pp. 10–11, Table 6 · p. 11

7 Ablation and Hyperparameter Study

The ablations show that safety contributions vary across expert types and layers, while attack strength depends on localization breadth and threshold.

7.1 Sparse vs. Shared Experts

  • Pruning sparse experts increases average ASR by 38.8%, versus 36.6% for shared experts, but shared-expert impact varies substantially by model.Shared experts are always active: pruning them yields 50.2% ASR in OpenPangu-Pro-MoE-72B and 42.2% in Qwen1.5-MoE-A2.7B-Chat; sparse experts matter more in some other models.Source: §7.1 · p. 11, Table 7 · p. 11

7.2 Gate-Projection vs. Up-Projection Layers

  • Pruning both expert MLP gate- and up-projection layers raises average ASR to 64.9%, compared with 55.9% for gate-only and 20.6% for up-only pruning.The gate-projection layer dominates, and the authors suggest its early, sparse, discriminative activations may support specialized safety responses.Source: §7.2 · pp. 11–12, Table 8 · p. 12

7.3 The Range of Safety Experts

  • Expanding the safety-expert range raises average ASR from 37.0% with Top-k to 51.3% with Top-2k and 64.9% with Top-3k.The authors report that safety behavior extends beyond the most frequently selected experts; Top-3k captures 2.6% of neurons per layer on average.Source: §7.3 · p. 12, Table 9 · p. 12

7.4 Impact of the z-Threshold

  • The default z-threshold τ = 2 balances attack strength and utility, while τ = 1 risks suppressing non-safety neurons and τ = 3 reduces ASR.DeepSeek-MoE and Mixtral show stable ASR across thresholds, whereas Qwen3-30B and GPT-OSS-20B are more sensitive.Source: §7.4 · p. 12, Table 10 · p. 12

8 Discussion

The discussion describes a proxy-model route to black-box prompt attacks and outlines possible MoE-specific defenses, while noting an assumption behind transfer.

Black-box Attack

  • A black-box prompt strategy would use a public MoE proxy to identify safety-linked routing, then craft prompts to avoid those experts in a target model.The approach assumes structural or alignment similarities between proxy and target, which the authors relate to sibling-model transfer attacks.Source: §8 · pp. 12–13

Potential Defenses

  • The authors propose distributing safety behavior across experts, embedding safety objectives in each expert, and monitoring or checksumming activations.They note that existing defense research mainly targets dense LLMs and that no widely adopted or evaluated defense specifically targets MoE LLMs.Source: §8 · p. 13

9 Related Works

Prior work addresses dense-model jailbreaks and neuron-targeted attacks, while MoE safety research remains limited; GateBreaker differs from coarse expert masking by targeting neurons.

9 Related Works

  • Dense-model jailbreaks progressed from fixed templates [53] [54] [55] to fuzzing [56], gradient-based optimization [57], and generative refinement [60] [61], but remain input-level attacks.Neuron studies [63] and safety-neuron removal work motivate internal, neuron-targeted approaches.Source: §9 · p. 13
  • MoE studies report attacks exploiting cross-batch routing [66], dormant-expert backdoors [27], and routing tie-break leakage [67], while safety analysis remains limited.These examples concern integrity, availability, backdoors, and side-channel leakage, rather than the fine-grained safety removal described for GateBreaker.Source: §9 · p. 13
  • SAFEx [28] masks presumed safety control experts, whereas GateBreaker localizes and removes safety-relevant neurons for finer-grained modification.The cited comparison says SAFEx assumes dedicated safety experts and operates at the coarse expert level.Source: §9 · p. 13

10 Conclusions

The conclusion reports GateBreaker’s attack results across MoE LLMs and VLMs, alongside transfer across model-family variants.

10 Conclusions

  • Across eight open MoE models, GateBreaker raises average ASR from 7.4% to 64.9% by removing 2.6% of neurons in relevant expert layers on average.It raises ASR from 20.8% to 60.9% across five MoE VLMs and achieves 67.7% ASR in one-shot transfer attacks across family variants.Source: §10 · p. 13

Ethical Considerations

The paper flags GateBreaker’s dual-use potential and frames its findings as diagnostic insight for strengthening MoE safety. It describes disclosure, dissemination, and participant-wellbeing measures intended to manage research risks.

Stakeholders

  • The study focuses on openly released MoE models, and the authors report notifying relevant stakeholders while following coordinated vulnerability disclosure practices.They say they will incorporate feedback and observe reasonable disclosure timelines where appropriate.Source: Ethical Considerations · p. 14

Potential Impact

  • GateBreaker could be misused to weaken or bypass safety alignment, while the authors frame the work’s goal as strengthening MoE security.They say they communicate its defensive and diagnostic purpose and release only reproducibility and evaluation components with a research-only disclaimer and usage guidelines.Source: Ethical Considerations · p. 14

Mitigations for Negative Impacts

  • Experiments used openly released models in controlled research environments, without human subjects, personal data, or interaction with deployed systems.Source: Ethical Considerations · p. 14

Decision to Conduct and Publish the Study

  • The authors cite a gap in MoE safety defenses relative to dense architectures as their reason for conducting and publishing the study.They argue that transparency, early disclosure, and defensive research benefits outweigh potential publication harms when paired with responsible disclosure and controlled dissemination.Source: Ethical Considerations · p. 14

Protection of Research Team Members

  • A human judge reviewed a small part of the research to rule out false positives in harmfulness detection, with safety protocols and support for team members exposed to harmful content.Source: Ethical Considerations · p. 14

Open Science

The authors make the study’s artifacts permanently available to the security research community. They describe the release as supporting replication and independent validation without large-scale computing infrastructure.

Open Science

  • All study artifacts, including code for gate-level profiling, expert-level localization, and targeted safety removal, are permanently available to the security research community.Source: Open Science · p. 14
  • GateBreaker targets openly released models and public datasets, enabling full replication of the methodology and results.Source: Open Science · p. 14
  • The pipeline is designed to run on consumer-grade GPUs so researchers can validate and extend the work without large-scale computing infrastructure.Source: Open Science · p. 14

Appendix

The appendix examines how expert ablation, suppression strength, and neuron-targeting specificity relate to attack success. Its results show model-specific patterns, stronger ASR with stronger suppression, and a large gap between targeted and random pruning.

A Sparse Expert Ablation

  • 45.9% average ASR at top-2K expert ablation, up from 19.3% at top-0.5K, with gains varying across models.GPT-OSS-20B rises from 15.0% to 73.8%; Mixtral-8x7B-Instruct-v0.1 already reaches 56.9% at top-0.5K and increases more modestly.Source: A Sparse Expert Ablation · p. 18, Table 11 · p. 18

B Safety Neuron Suppression Strength

C The Importance of Safety Neurons

  • 64.9% average ASR with GateBreaker, versus 7.8% and 11.8% for the two random-pruning baselines.The authors interpret the contrast as evidence that the jailbreak success is not due to reduced model capacity or degraded fluency, but to disrupting safety-relevant representations.Source: C The Importance of Safety Neurons · pp. 18–19, Table 13 · p. 18
Loading 2512.21008v2…