Tripwire: Triggering Aligned Refusal via Statistically Certified Safety Neurons

Wei Zhao; Zhe Li; Peixin Zhang; Jun Sun

Source-linked AI summary

Tripwire: Triggering Aligned Refusal via Statistically Certified Safety Neurons

Wei Zhao, Zhe Li, Peixin Zhang, Jun Sun

arXiv:2608.14392v1Aug 14, 2026cs.AI

TL;DR

Existing neuron-level jailbreak defenses can protect safety at substantial utility cost because they erase distributed harmful semantics, misidentify utility-critical neurons, or perturb benign requests. TripWire statistically identifies safety-specific neurons and trigger-clamps them, achieving below 2% average attack success rate with the smallest utility drop among evaluated defenses.

Problem
Existing neuron-level defenses struggle to preserve utility because harmful semantics are distributed, safety neurons can overlap with generally important neurons, and interventions may always perturb benign requests.
Method
TripWire uses false-discovery-rate-controlled statistical selection and utility-specificity filtering to identify safety-specific neurons, then clamps their activations at harmful-conditional means.
Results
Below 2% average attack success rate was achieved across four safety-aligned LLMs and four representative attacks, with the smallest utility drop among defenses.
Takeaways & Limitations
TripWire supports practical neuron-level jailbreak protection without costly retraining by combining precise neuron identification with trigger-style intervention.
Takeaways & Limitations
The false-discovery-rate guarantee depends on the standard assumptions of the Benjamini–Hochberg procedure.

Abstract

from arXiv · show

Neuron- and path-level interventions offer the finest-grained route to defending large language models (LLMs) against jailbreak attacks, yet existing methods fall short of this promise, i.e., they often compromise model utility significantly. Specifically, one line of work suppresses toxic neurons to erase harmful semantics, but since such semantics are distributed across the network, blocking every pathway forces a large intervention footprint. An alternative line of research focus on identify safety neurons using external classifiers. While promising, the existing approaches suffer from compromising neurons that are important for the model utility as well. Moreover, both approaches remain always on and thus perturb every benign request even when no attack is present. To address these limitations, we present \ours{}, a training-free defense that first identifies safety-specific neurons through per-neuron hypothesis tests under false-discovery-rate control together with a utility-specificity filter. Based on this identification, a trigger-style clamp holds the selected neurons at their harmful-conditional mean activations, injecting an internal harmful-input signal that triggers the refusal behavior learned during alignment. The clamp is then realized by two provably equivalent deployment modes, namely a detector-gated inference-time intervention and an offline bias-patch weight edit. Extensive experiments across four safety-aligned LLMs and four representative attacks demonstrate that \ours{} reduces the average attack success rate to at most 2.0\% while incurring a utility drop of only 0.5\% to 5.3\% on MT-Bench, the smallest among all defenses. Code is available at https://anonymous.4open.science/r/Tripwire-65C4.

Introduction

Jailbreaking threatens the security and reliability of deployed LLMs, while existing defenses struggle to improve refusal behavior without degrading benign-task utility. TripWire addresses this trade-off with statistically filtered neuron selection and evaluation across multiple aligned models and jailbreak attacks.

Jailbreaking is a severe threat because deployed LLM vulnerabilities create significant security and reliability risks.
Existing defenses face a persistent safety–utility trade-off: stronger refusal behavior degrades performance on benign tasks.
Toxic-neuron defenses require suppressing distributed harmful-semantic pathways, producing a large intervention footprint that perturbs every additional input.
TripWire screens every neuron using direction filtering, Welch t-tests with Benjamini–Hochberg false-discovery-rate control, and a utility-specificity filter before selecting top-N candidates.The funnel removes neurons that also activate strongly on normal tasks and provides a statistical upper bound on mis-selection.
TripWire is evaluated on four safety-aligned open-source LLMs against four representative jailbreak attacks and compared with inference-based and knowledge-editing defense baselines.The models are Llama-2-7B, Llama-3.1-8B, Qwen2.5-7B, and Qwen2.5-32B; attacks include GCG, AmpleGCG, AutoDAN, and Jailbreak-R1.

Preliminary

The paper models decoder-only Transformers through their MLP sublayers, defining neurons by projection and analyzing last-token activations across harmful, benign, and utility prompts. It motivates a statistically controlled identification funnel by highlighting limitations of classifier-based safety-neuron screening.

Architecture: Decoder-only Transformers autoregressively map token sequences to next-token distributions through stacked attention and MLP sublayers.Attention exchanges information across token positions, while MLPs transform each position independently.
Architecture: The analysis centers on MLPs because they contain most model parameters and are widely regarded as a primary store of knowledge.Modern Llama and Qwen models implement these sublayers in gated form.
Neuron representation: A neuron is defined as a triple (ℓ, ψ, i), with ψ ∈ {gate, up} and i indexing an intermediate projection.Each prompt is represented by its last-token activation, whose position aggregates the prompt’s semantic information through causal attention.
Neuron representation: 200 prompts each from HarmBench, Alpaca, and MMLU provide harmful, benign, and utility activation sets for analysis.The sets are denoted Dh, Ds, and Du, respectively.
Motivation: Classifier-based screening can select correlated rather than safety-specific neurons, leaves false selections uncontrolled, and may reduce utility without defensive gain.Existing work trains per-layer logistic-regression probes and selects neurons by thresholded weight magnitude, but probe weights are not identifiable when p ≫ n.

Method

TripWire identifies safety-specific neurons through directional, statistically significant, utility-specific filtering under FDR control, then clamps selected neurons to harmful-conditional means. It deploys this clamp either conditionally through a malicious-request detector or permanently through an equivalent bias-patch weight edit.

Safety-neuron identification: TripWire screens neurons using direction, Welch t-test significance with Benjamini–Hochberg FDR correction, and utility-specificity filters before selecting a top-N budget.Direction requires μ_h > μ_s; the FDR procedure controls the expected fraction of irrelevant selected neurons, while specificity filtering targets over-refusal and utility protection.
Safety-neuron identification: The utility-specificity filter excludes neurons that also activate strongly on normal utility tasks, using the threshold ρ > τρ with default τρ = 0.5.ρ ≈ 0 corresponds to benign-like utility behavior, whereas ρ ≈ 1 indicates generally important rather than safety-specific neurons.
Trigger-style clamp: The selected neurons are clamped at their harmful-conditional mean activations throughout generation, injecting a harmfulness signal rather than erasing every harmful-activation pathway.This contrasts with erasure-style interventions that attempt to suppress all routes through which distributed harmful semantics propagate.
Detector-gated deployment: A logistic-regression detector uses last-token activations from the selected neurons and applies the clamp only when a request is judged malicious.Benign requests are generated by the unmodified model; detector training reuses the benign set D_s and harmful set D_h used for identification.
Bias-patch deployment: The bias-patch realizes the always-on clamp by zeroing each selected neuron’s weight row and injecting its harmful-conditional mean as the bias.This provides a hardened, detector-free checkpoint for architectures with or without an existing bias parameter.
Bias-patch deployment: The bias-patched checkpoint and always-on inference-time clamp compute mathematically identical maps for every input and token position, differing only in permanence and detector use.Thus, the deployment choice is operational rather than a difference in the defense map.

Experiment

Experiments across four safety-aligned LLMs and four jailbreak attacks show that TripWire substantially reduces attack success while preserving utility better than competing defenses. Ablations and layer analyses support the funnel’s safety–utility advantage and reveal where selected safety neurons are concentrated.

Experimental setup: Experiments use four safety-aligned LLMs and four representative jailbreak attacks generated from AdvBench’s 520 harmful prompts.The models are Llama-2-7B, Llama-3.1-8B, Qwen2.5-7B, and Qwen2.5-32B; attacks are GCG, AmpleGCG, AutoDAN, and Jailbreak-R1.
Defense results: TripWire at top-2500 reduces average ASR from 43.0% to 1.1% on Llama-2, from 33.1% to 0.5% on Llama-3.1, and from 24.1% to 2.0% on Qwen2.5.The corresponding utility drops are 5.3%, 4.7%, and 0.5%, respectively.
Defense results: The top-1000 inference configuration keeps utility drop within 3.6% and over-refusal at most 2.5% of benign queries.The detector leaves the benign path untouched, while the ASR at this budget is slightly higher.
Defense results: TripWire at top-2500 achieves average ASR below 2% across both deployment forms, versus 8.3%–12.6% for RepE and 2.2%–4.9% for TraceRouter, DELMAN, and LED.It also incurs the smallest utility drop among all defenses.
Identification ablation: Probe-based selection requires 3–5× more neurons than TripWire’s funnel for comparable ASR, yet degrades utility by up to 52.3%.TripWire’s inference-without-detector and edit forms yield nearly identical ASR and MT-Bench scores at both budgets, empirically confirming their equivalence.
Neuron distribution: Across all three models, selected safety neurons concentrate in middle-to-upper layers and are nearly absent from early layers.Llama-2 and Llama-3.1 peak around layers 12–16, whereas Qwen2.5-7B concentrates mainly in layers 15–21 with onset around layer 13.

Related Work

Related work spans jailbreak attacks and defenses organized by intervention granularity. Existing defenses range from whole-model alignment and adversarial training to finer-grained neuron-level approaches, while jailbreaks use optimization or generator-based methods to elicit harmful outputs.

Jailbreak attacks: Optimization-based attacks use gradient signals to search for adversarial suffixes that maximize harmful-response probability.
Jailbreak attacks: Generator-based attacks fine-tune models on successful jailbreaks, enabling diverse adversarial prompts.
Defense granularity: Alignment and adversarial training instill refusal by shifting the entire response distribution, but must be repeated for every emerging threat.
Defense granularity: Existing defenses are organized by intervention granularity, ranging from the whole model down to individual neurons.

Conclusion

The conclusion identifies distributed harmful semantics, imperfect safety-neuron attribution, and always-on perturbations as the key failures of existing neuron-level defenses. It presents TripWire as a training-free statistical-neuron clamp with equivalent deployment modes that reduces attack success across tested models and attacks.

Limitations of existing defenses: Existing defenses fail because harmful semantics are distributed across the network, safety neurons are difficult to distinguish from generally important neurons, and interventions perturb benign requests.The conclusion attributes these shortcomings to erasure-style methods, classifier-based attribution, and always-on intervention designs.
TripWire: TripWire identifies safety-specific neurons with false-discovery-rate control and clamps their harmful-conditional mean activations to trigger aligned refusal behavior.The training-free defense uses a statistical funnel and realizes the clamp through two provably equivalent deployment modes.
Evaluation: 2.0%: TripWire reduces average attack success rate to at most 2.0% across four safety-aligned LLMs and four representative attacks.The reported evaluation covers four models and four attacks.