NeuronGuard: Robust LLM Safety Alignment via Ablation-Aware Safety Signal Redistribution

Anjun Gao; Yueyang Quan; Yufei Xia; Zhuqing Liu; Minghong Fang

Source-linked AI summary

NeuronGuard: Robust LLM Safety Alignment via Ablation-Aware Safety Signal Redistribution

Anjun Gao, Yueyang Quan, Yufei Xia, Zhuqing Liu, Minghong Fang

arXiv:2608.23959v1Aug 25, 2026cs.CRcs.AIcs.IRcs.LG

TL;DR

LLM safety alignment remains vulnerable to jailbreaks and neuron-level attacks because safety information is concentrated in sparse neuron subsets. NeuronGuard redistributes safety signals during fine-tuning through dynamic neuron identification, ablation-robust training, and randomized gradient correction. It reports near-zero ASR across diverse, multimodal, and adaptive attack settings while preserving task utility, but requires control of the fine-tuning pipeline.

Problem
LLM alignment remains fragile against jailbreak and neuron-level attacks that exploit concentrated safety information and can bypass or suppress safety mechanisms.
Method
NeuronGuard dynamically identifies safety-critical neurons, trains refusal behavior under their ablation, regularizes output consistency, and resolves safety-task gradient conflicts during fine-tuning.
Results
NeuronGuard achieves near-zero ASR across diverse attack settings, multimodal scenarios, and white-box adaptive adversaries while preserving task utility.
Takeaways & Limitations
Redistributing safety signals at the neuron level provides a unified fine-tuning-stage defense against jailbreak and neuron-level attacks within the evaluated settings.
Takeaways & Limitations
NeuronGuard assumes defender control over training data, weights, and optimization, so complementary inference-time defenses are needed when such control is unavailable.

Abstract

from arXiv · show

Safety alignment in large language models (LLMs) remains brittle against a growing spectrum of attacks. Jailbreak attacks bypass safety mechanisms through crafted prompts, while neuron-level attacks directly prune safety-critical neurons post-deployment. Both exploit a common weakness: safety-relevant information concentrates in a sparse neuron subset. We present NeuronGuard, a fine-tuning-stage defense that simultaneously hardens LLMs against both attack classes by redistributing safety signals across a broader set of neurons. NeuronGuard dynamically identifies safety-critical neurons via periodically refreshed per-layer linear classifiers, forces refusal behavior under deliberate neuron ablation, and applies KL-divergence regularization for distributional consistency. A randomized gradient projection strategy preserves downstream task utility by resolving conflicts between the defense and task objectives. We provide a formal guarantee that NeuronGuard strictly reduces the attack success rate (ASR) upper bound, and experiments across three LLMs, six state-of-the-art attack strategies, and multimodal settings confirm near-zero ASR while maintaining task accuracy, including against white-box adaptive adversaries.

1 Introduction

NeuronGuard addresses brittle alignment by redistributing safety information beyond sparse safety-critical neuron subsets, targeting both jailbreak and neuron-level attacks. It combines dynamic neuron identification, ablation-robust safety training, and conflict-aware optimization while preserving downstream utility.

Alignment remains vulnerable to crafted-prompt jailbreaks and parameter-level neuron attacks that suppress safety-critical internal components.
Existing prompt- and model-level defenses remain inadequate because they do not address safety information concentrated in sparse neuron subsets.
NeuronGuard periodically refreshes per-layer linear classifiers to identify safety-critical neurons as model representations evolve.
Evaluations across six attack strategies, multimodal settings, and adaptive adversaries report near-zero ASR while maintaining task accuracy, with modest computational overhead.
Ablation-robust optimization trains the model to refuse harmful queries both normally and after identified safety neurons are zeroed out, promoting distributed safety encoding.
Randomized gradient correction resolves conflicts between user-task and safety objectives, preventing destructive interference while preserving task utility and safety robustness.

5 Theoretical analysis

NeuronGuard’s theoretical analysis models how safety-neuron identification, ablation coverage, attack capability, output sensitivity, and KL divergence determine robustness. Under stated assumptions and a sufficient regularization condition, the defense strictly lowers the ASR upper bound relative to an undefended model.

Assumptions and quantities: The analysis defines safety-critical neurons using activation differences between unsafe and safe inputs, with a sparse ground-truth set at each layer.The set is characterized by a thresholded expected activation gap, with |S_l| much smaller than the layer width.
Assumptions and quantities: The theory assumes approximate linear separability, sufficient ablation coverage, bounded attacker perturbations, local output-head Lipschitz continuity, and bounded KL divergence.The attacker is restricted to safety neurons, while ablation removes part of the exploitable perturbation support.
Proof strategy: The proof uses three auxiliary steps: ablation shrinks the adversarial subspace, Lipschitz continuity controls output perturbations, and Pinsker’s inequality converts bounded KL divergence into bounded ℓ1 distance.The condition on KL regularization captures a trade-off between reduced exploitability and distributional shift caused by ablation.
ASR bound: The theorem compares undefended and defended neuron-exploitation probabilities using per-neuron attack success probability and ablation coverage.The undefended probability is 1−(1−α_l)^s_l, whereas the defended probability is 1−(1−α_l)(1−ρ_l)^s_l.
ASR bound: NeuronGuard strictly reduces the ASR upper bound when the KL-regularization strength satisfies the theorem’s sufficient condition.The result is expressed as B(ASR_nl) ≥ B(ASR_ours), with strict reduction relative to the undefended model under the stated assumptions.

6 Experiments

Experiments evaluate NeuronGuard across three LLMs, four fine-tuning tasks, six attacks, and varied settings, finding near-zero ASR with utility close to no defense and modest overhead.

Experimental setup: Evaluations cover three pretrained LLMs, four utility datasets, six attack strategies, and complementary accuracy and attack-success metrics.The attacks span generation-based and optimization-based jailbreaks alongside neuron-level attacks.
Utility: NeuronGuard preserves high ACC across four tasks and closely matches the No defense setting.On Falcon (SST2), it achieves 0.93 ACC versus 0.94 under No defense; on Llama (AGNews), 0.90 versus 0.91.
Robustness: NeuronGuard reduces ASR to nearly zero across attacks, achieving 0.00 under PAIR on Qwen and 0.04 under NeuroStrike on Llama.The reported comparisons cover SST2, with additional ASR tables for AGNews, CoLA, and GSM8K.
Sensitivity analyses: Performance remains stable across different ablation ratios, classifier refresh intervals, and safety-probe dataset sizes.The analyses report robust behavior at low ρ, across a wide range of N, and with as few as 500 samples.
Efficiency: NeuronGuard’s fine-tuning overhead remains on the same order of magnitude as No defense and adds no inference-time cost.This differs from test-time defenses that require extra operations per input.

7 Discussion

Discussion analyses component contributions, adaptive attacks, multimodal transfer, dynamic neuron identification, and loss weighting, with robustness and utility broadly preserved across these tests.

Ablation analysis: Component ablations compare refusal training, neuron ablation, consistency regularization, and randomized gradient projection to isolate their contributions.The variants differ in whether they include Lcons, gradient projection, or simultaneous rather than randomized projection.
Adaptive attacks: Adaptive evaluations include iterative pruning and nonlinear evasion attacks that assume full knowledge of NeuronGuard.These attacks respectively search progressively pruned models and use nonlinear models to identify safety-relevant neurons.
Multimodal evaluation: In multimodal settings, NeuronGuard matches no-defense ACC and outperforms all baselines in ASR under image-based attack settings.The evaluation uses Qwen2.5-VL-7B-Instruct, T2I-formatted SST2 data, StrongREJECT queries, and NSFW images.
Dynamic identification: Replacing dynamic neuron identification with SafeNeuron’s static strategy increases ASR under NeuroStrike on Llama.The comparison keeps the remaining NeuronGuard components unchanged.
Loss weighting: Across tested loss weightings, NeuronGuard keeps ASR at most 0.08 under all six attacks, while larger λ_safe trades accuracy for marginal ASR gains.The sensitivity analysis varies λ_safe to 0.5 and 2.

8 Conclusion

NeuronGuard redistributes safety signals across neurons through ablation-robust optimization and related mechanisms, achieving near-zero ASR while preserving task utility across evaluated settings.

Conclusion: Ablation-robust optimization, KL-divergence regularization, and randomized gradient projection jointly target jailbreak and neuron-level attacks while preserving task utility.The conclusion reports this scope across diverse attack settings, multimodal scenarios, and white-box adaptive adversaries.
Conclusion: The reported findings achieve near-zero ASR across diverse attack settings, multimodal scenarios, and white-box adaptive adversaries.The paper concludes by emphasizing neuron-level safety robustness.

9 Limitations

The paper’s evaluation and method description emphasize a fine-tuning pipeline with staged neuron identification, ablation-based redistribution, consistency regularization, and gradient projection, while deployment scope is constrained by training control.

Limitations: NeuronGuard assumes defender control over training data, model weights, and optimization objectives during fine-tuning.This assumption may fail when models are deployed without further fine-tuning, resources are limited, or access is only black-box API access.
Limitations: The paper notes that complementary inference-time defenses would be needed when fine-tuning control is unavailable.This boundary follows from the method operating during fine-tuning.
Method overview: Stage 1 trains per-layer classifiers to identify safety-critical neurons, and Stage 2 ablates them during training while enforcing consistency between passes.The supplied procedure references classifier training, neuron-set identification, refusal loss, and consistency loss.
Method overview: Stage 3 resolves conflicts between safety and user-task gradients through randomized projection, producing a final update intended to preserve robustness and utility.The figure describes the gradients as gsafe and guser and the resulting update as gfinal.

A Discussion of assumptions

The discussion frames the theory around approximate linear separability of safety behavior, coverage of the true safety-neuron set, bounded KL divergence, and reduced effective attack subspaces after ablation.

Assumptions: Assumption 1 treats safety-related activations as approximately linearly separable at selected layers, especially for models with stable refusal patterns.This relies on sparse specialization of high-level behaviors and activation shifts between benign and unsafe prompts.
Assumptions: The theoretical coverage ratio ρ_l measures overlap between probe-selected safety neurons and the true safety-neuron set, unlike hyperparameter ρ, which controls selection size.The empirical set I_safety,l is obtained through top-ρ linear-classifier selection.
Assumptions: KL regularization uses the standard model distribution as reference, enforcing prediction consistency while maintaining refusal behavior on unsafe prompts.The safety objective also uses a human-defined refusal reward, and theoretical unsafe and safe populations correspond to empirical datasets.
Ablation effects: Ablation restricts effective perturbations to S_eff = S_l \\ I_safety,l, because perturbations on ablated coordinates are nullified.The remaining effective attackable subspace has size (1 − ρ_l)|S_l|.
Ablation effects: Ablation shrinks the adversarial subspace by a factor of (1 − ρ_l).The result follows from the definition of the ablation coverage ratio.
Attackability: For per-neuron attack probability α_l, reducing available neurons from |S_l| to (1 − ρ_l)|S_l| strictly lowers successful neuron-level exploitation probability.The comparison assumes independent per-neuron attack attempts and ρ_l, α_l ∈ (0,1).
Output control: The Lipschitz assumption bounds output changes caused by admissible perturbations around the ablated activation.The ablation operation itself is incorporated into the ablated activation vector.
Distributional consistency: An expected KL bound yields a small expected ℓ1 difference between standard and ablated output distributions via Pinsker’s inequality and Jensen’s inequality.The derivation uses E_x∼Dunsafe[Z(x)] ≤ η.

B.2 Proof for Theorem 1

The theorem proof compares ASR upper bounds for NeuroStrike and NeuronGuard by combining neuron-exploitation probabilities, Lipschitz output control, and distributional consistency after ablation.

ASR bounds: The NeuroStrike bound allows exploitation of all s_l = |S_l| safety neurons and adds a Lipschitz-based clean-margin failure term.The bound uses a union bound over neuron exploitation and insufficient clean safety margin.
ASR bounds: NeuronGuard reduces the effective exploitable population to (1 − ρ_l)s_l and bounds unsafe generations through the ablated model’s unsafe-output probability.The resulting bound is further related to the standard model through total variation distance.
Bound comparison: The difference between the two bounds combines reduced neuron exploitation with the expected unsafe-output probability under the ablated model.The proof obtains this expression by subtracting the two ASR upper bounds.
Bound comparison: Because (1 − ρ_l)s_l < s_l and (1 − α_l)^t decreases strictly with t, the defense bound is strictly smaller than the NeuroStrike bound.The argument assumes ρ_l and α_l lie in (0,1).

C License of models and datasets

The paper documents licenses for its models and datasets, including custom, Apache, MIT, and research-use terms.

Models: Llama-3.1-8B-Instruct and Llama-Guard-3-8B use Meta’s Llama 3.1 Community License.Qwen2.5-7B-Instruct and Qwen2.5-VL-7B-Instruct are identified as Apache-licensed in the passage.
Datasets: GSM8K, StrongREJECT, CoLA, and the NSFW Detection dataset are released under MIT-compatible licensing terms described in the paper.AGNews is described as research-oriented without an explicit open-source license.

D Details of baselines

The baselines cover input filtering, prompt perturbation, gradient analysis, adversarial training, layer editing, and neuron freezing to address jailbreak or neuron-pruning attacks.

Input-level defenses: Perplexity flags potential jailbreak attempts using the model’s prompt perplexity against a threshold.It operates as an input filter.
Input-level defenses: SmoothLLM perturbs prompts at the character level, aggregates multiple responses, and uses majority voting without model retraining.Its premise is that adversarial prompts are fragile to random perturbations.
Gradient-based defense: GradSafe detects jailbreak prompts from consistent gradients on safety-critical parameters that differ from safe-prompt gradients.The method pairs prompts with compliance responses during gradient analysis.
Adversarial training: CAT adversarially trains in continuous embedding space after selecting effective perturbations from K candidates with a lightweight surrogate model.The selected perturbations are mapped back to discrete tokens for fine-tuning.
Model editing: LED identifies and edits safety-critical layers using hidden-state differences between safe and adversarial prompts.It is an editing-based jailbreak defense.
Neuron-level defense: SafeNeuron identifies safety-critical neurons with activation-based metrics and freezes them during direct preference optimization to encourage redundant safety pathways.Its target is robustness against neuron-level pruning attacks.

E Details of adaptive attacks

The adaptive attacks target different limits of NeuronGuard’s defense: iterative pruning repeatedly removes newly identified safety neurons, while nonlinear evasion searches for neurons involved in nonlinear interactions.

Iterative pruning: Iterative pruning retrains per-layer linear classifiers after each ablation round and permanently removes the newly identified safety neurons.The attack mirrors NeuronGuard’s identification procedure while progressively weakening the model.
Adversarial objectives: Both attacks adapt their neuron-selection strategy to exploit potential gaps in NeuronGuard’s safety-neuron identification.Iterative pruning uses repeated linear re-identification, whereas nonlinear evasion replaces linear classifiers with MLPs.
Nonlinear evasion: Nonlinear evasion trains a two-layer MLP for each layer to identify safety-neuron sets that linear classifiers may miss.It targets safety-relevant neurons whose importance arises from nonlinear neuron interactions.

F Robustness under adaptive attacks

NeuronGuard’s adaptive robustness relies on distributing safety signals across an expanding neuron population, so attackers cannot expect fixed-target suppression to remain effective. The evaluation examines adaptive attacks, multimodal settings, identification choices, masking, refresh intervals, data sizes, and defense variants across multiple tasks and models.

Adaptive robustness: Safety redistribution prevents a fully informed attacker from achieving lasting success by suppressing only a fixed, locatable neuron set.During fine-tuning, safety signals spread across an increasingly broad population of neurons.
Adaptive robustness: Iterative pruning must eliminate safety behavior across all redundant sites, whereas NeuronGuard only needs to preserve it in a sufficient fraction of them.This asymmetry supports resilience even when the attacker repeatedly applies the defense’s identification logic.
Adaptive robustness: Partial ablation coverage is sufficient to shrink the neurons available for exploitation, because robustness does not require exhaustive neuron identification.The nonlinear evasion analysis also connects this resilience to KL-divergence regularization.
Evaluation scope: The adaptive-attack evaluation includes ASR comparisons across six attack types and three models on AGNews, CoLA, and GSM8K.The reported tables organize results by task, attack type, model, and defense method.
Evaluation scope: Additional analyses examine adaptive attacks, multimodal settings, static versus dynamic identification, masking fractions, refresh intervals, data sizes, and NeuronGuard variants.These studies are reported for SST2 in the listed tables.