PaperlayerView exact revision on arXiv

Source-linked AI summary

Do VLMs Share Safety Neurons Across Modalities?

Jiaxuan Li, Jiahao Zhang, Duc Minh Vo, Huy H. Nguyen, Pride Kavumba, Koki Wataoka

arXiv:2608.30750v1cs.LG

TL;DR

The paper examines how visual inputs alter safety pathways in VLMs at the neuron level, a gap left by prior empirical and representation-level analyses. It uses causal neuron analysis, iterative ablation, and modality-isolated benchmarks across 10 VLMs, finding localized dominant text safety but diffuse visual safety.

  • Problem

    Neuron-level evidence about how visual inputs perturb VLM safety pathways is limited, despite image-based harmful requests reopening the attack surface.

  • Method

    The study applies a two-stage causal detection pipeline with iterative ablation across 10 VLMs and introduces ViSafe-Detect and ViSafe-Eval to isolate visual and textual safety signals.

  • Results

    Text safety localizes to approximately 88 neurons, while text neurons are the only intervention consistently and substantially reducing refusal across all image conditions; visual safety is diffuse and high-dimensional.

  • Takeaways & Limitations

    The results indicate that visual safety cannot be addressed by assuming a comparably localized text gate, motivating alignment signals applied directly to visual representations.

  • Takeaways & Limitations

    The interventions target only the LLM backbone, while the analysis also uses narrow harmful-image and intent datasets and one judge model.

Abstract

from arXiv · show

Vision-language models (VLMs) can comply with harmful requests delivered through images, even when their LLM backbones would refuse the same content in text. While prior work characterizes these jailbreaks empirically or at the representation level, how visual inputs perturb safety pathways at the neuron level remains uncharted. We close this gap with a causal, neuron-level analysis of safety mechanisms in 10 VLMs. We propose a two-stage detection pipeline with iterative ablation that accounts for self-repair, and introduce two modality-isolated benchmarks, ViSafe-Detect and ViSafe-Eval, which decouple visual and textual safety signals. Our analysis reveals: (i) Text safety in VLMs is localizable: $\sim$88 neurons ($<$0.01%) whose targeted ablation substantially reduces refusal. (ii) Text safety neurons constitute the dominant refusal pathway: ablating them is the only intervention that consistently and substantially reduces refusal across all models. (iii) Visual safety is high-dimensional and diffuse at the single-neuron level: text safety concentrates in $\sim$5 subspace directions while visual safety requires $\geq$50. This gap holds across architectures, explaining why current alignment has not closed the visual safety gap. Project page is at: https://jiaxuan-li.github.io/vlm-safety-neuron/ Warning: this paper may include examples of harmful content.

1 Introduction

This paper investigates whether VLMs use the same localized safety neurons for visual and textual harm, using causal neuron-level analysis across 10 models. It finds localized, dominant text safety but diffuse visual safety, supported by a self-repair-aware pipeline and modality-isolated benchmarks.

  • Motivation: VLMs can comply with harmful requests delivered through images, while the same harmful image is largely ignored when the query is benign.This motivates examining how visual inputs interact with safety mechanisms inside the shared language backbone.
  • Research gap: Prior work localizes text safety or analyzes visual distortions at the representation level, but does not identify neuron-level mechanisms inside the shared decoder.The paper frames this as the unresolved question of whether text and visual harm use the same safety neurons or separate mechanisms.
  • Approach: A two-stage detector combines behavioral probing, mean-activation patching, and three-round iterative ablation to account for self-repair.The same confirmation criteria are applied to text and visual stimuli, while re-detection exposes backup neurons hidden during earlier rounds.
  • Findings: Across 10 VLMs, text safety is localizable and dominant, whereas visual safety is diffuse and requires at least 50 representation-level directions.On average, ∼88 text neurons account for refusal; text safety saturates within ∼5 directions, while visual safety is not captured by a comparably small single-neuron set.

2 Related Work

The paper builds on mechanistic studies of sparse text safety and representation-level multimodal analyses, extending them with causal neuron-level analysis inside VLM backbones. Its contribution is a dimensionality comparison showing sparse text safety versus distributed visual safety.

  • Text safety mechanisms: Prior mechanistic studies show that text safety can be disrupted by perturbing a small fraction of model parameters, neurons, or a single residual-stream direction.These findings motivate searching for sparse safety structure in VLM decoders.
  • Mechanistic methodology: The proposed framework adapts probe-based screening with mean-activation patching, BH-FDR control, and iterative ablation to expose backup neurons.It extends this analysis from LLMs to VLMs sharing a single decoder.
  • Multimodal safety: Prior multimodal safety studies document typographic, optimized, natural-image, and compositional jailbreaks but generally treat the backbone as a black box.The present work instead traces neuron-level mechanisms in the shared language backbone.
  • Contribution: The paper presents a causal neuron-level VLM analysis identifying a dimensionality gap between sparse text safety and distributed visual safety.This addresses a gap left unresolved by representation-level methods.

3 Method

The method detects safety neurons causally by combining behavioral screening, statistically controlled mean-activation patching, and iterative re-detection to expose self-repair backups.

  • Motivation: Self-repair makes single-pass detection incomplete because silencing one neuron allows parallel safety neurons to compensate.The pipeline therefore aims to remain scalable and statistically valid while recovering the safety-relevant population.
  • Behavioral Probing: Behavioral probing labels examples by model behavior rather than content, reducing the risk of selecting neurons correlated with lexical or visual cues.Keyword matching supplies first-pass labels, while GPT-4o re-judges responses classified as COMPLY to capture subtle refusals.
  • Behavioral Probing: |z_n| > 3.0 retains 1,800–2,300 candidates per model, producing an approximately 400× reduction before causal verification.Scores are standardized within each layer and candidates are ranked by |z_n| for fixed-budget verification.
  • Activation Patching: Mean-activation patching replaces a candidate activation with its empirical safe-query mean to test whether refusal tendency decreases under a safe-like counterfactual.This avoids the out-of-distribution signal introduced by zero-ablation.
  • Activation Patching: BH-FDR correction against a control-neuron null confirms neurons only when adjusted p < 0.05, positive causal effect, and probe–patching sign agreement.The null uses 300 control neurons sampled proportionally to per-layer candidate density.
  • Iterative Ablation: Three iterative rounds re-run probing and patching after previously confirmed neurons are clamped to safe means, revealing backup neurons masked by self-repair.Previously confirmed neurons are excluded from later candidate pools, with early stopping below 20 behavioral positives.

4 Experimental Setup

The experiments evaluate 10 VLMs across six architecture families using modality-isolated safety datasets, varied image conditions, and refusal and capability metrics.

  • Models: 10 VLMs from 6 architecture families span 2B–32B parameters, including adapted and natively multimodal models.The study includes six adapted models and four natively multimodal models.
  • Data Composition: The VISAFE-DETECT harmful-image set is dominated by real images after quality filtering.The table describes the composition of the harmful-image sources.
  • Benchmarks: VISAFE-DETECT pairs every image with the same neutral procedural query to isolate visual safety signals from textual confounds.VISAFE-DETECT and VISAFE-EVAL use different image sources for out-of-distribution evaluation.
  • Benchmarks: ∼11,300 samples per model combine three query types with eight image conditions, including harmful, safe, blank, noisy, absent, and rendered-text images.The query types include direct harmful prompts, unrelated safe queries, and the neutral procedural template.
  • Evaluation: RR-Judge uses GPT-4o to label responses REFUSE or COMPLY, while RR-KW provides first-pass behavioral labels and a fast refusal indicator.Capability under intervention is assessed with perplexity on held-out Dolly-15k instructions, 5-shot MMLU, and MMMU.

5 Experiments and Results

Across 10 VLMs, refusal behavior depends primarily on textual safety pathways: text safety is sparse, causal, and robust, whereas visual safety is weaker, less transferable, and distributed across more directions.

  • 5.1 Refusal Behavior Analysis: Refusal depends primarily on the text query rather than image content across VISAFE-EVAL’s 10 models and image conditions.In the direct-harmful panel, swapping harmful, blank, or absent images barely changes refusal for most models; rendered-text images can reactivate textual safety through OCR.
  • 5.2 Text Safety Neurons: 48–130 text safety neurons per model account for substantial refusal reductions while remaining a tiny fraction of model neurons.Joint ablation reduces in-distribution refusal by 9–33%, and the identified neurons generalize to out-of-distribution harmful queries.
  • 5.2 Text Safety Neurons: Iterative detection is essential because the first round captures only 38–78% of text safety neurons.Later rounds expose backup neurons masked by dominant ones; counts do not scale with model size.
  • 5.4 Text vs. Visual Safety Neurons: 43% average refusal reduction follows text-neuron ablation under direct harm, versus approximately 1% after visual-neuron ablation.For visually isolated image-only harm, text ablation reduces refusal by 20%, while visual ablation changes it by a sign-inconsistent mean of +5%.
  • 5.4 Text vs. Visual Safety Neurons: k≤5 directions nearly saturate the text effect, whereas the visual effect grows through k=50, indicating higher-dimensional visual safety.Across models with measurable visual baselines, visual effects grow by 9–35 points from k=5 to k=50, while text effects change by only 1–5 points; the asymmetry survives random-direction adjustment.

6 Discussion

Text safety concentrates in transferable neurons and a small subspace, whereas visual safety is diffuse across many directions. This asymmetry motivates visual-side interventions that do not assume a sparse text-like gate.

  • ∼88 transferable text-safety neurons saturate within ∼5 directions, while visual safety effects grow through 50 directions.The visual pattern is diffuse rather than absent.
  • The post-fusion asymmetry appears in both adapted and natively multimodal models.
  • The results indicate a localizable, dominant text refusal pathway but no comparably localized visual gate under the tested interventions.
  • The findings motivate interventions that avoid assuming text’s sparse organization, including visual checkpoints and broader visual safety data.

7 Conclusion

Across 10 VLMs, text safety localizes to a tiny, transferable neuron set and dominates refusal, while visual safety spans many representation-level directions without a comparable localized gate.

  • ∼0.01% of neurons encode text safety, generalize out-of-distribution, and form the dominant text refusal pathway.
  • Visual safety spans ≥50 representation-level directions without a comparably localized gate.
  • The asymmetry appears in adapted and natively multimodal architectures, with safety decisions relying predominantly on text.
  • Closing the visual-side gap will likely require alignment signals applied directly to visual representations rather than depending on the text gate to propagate.

Limitations

The study identifies four limitations: intervention scope, detection-data diversity, the selected refusal regime, and reliance on a single judge model.

  • The interventions target only the LLM backbone, leaving safety dynamics in vision encoders, projection layers, attention, routing, and multi-neuron interactions unexamined.
  • BeaverTails-V supplies 82.5% of retained VISAFE-DETECT harmful images, limiting detection-data diversity and potentially weakening the generalization claim.
  • The factorial analysis uses 472 harmful intents selected for high text-only refusal consensus, so conclusions apply to a refusal-active regime.
  • RR-Judge relies on one judge model, whose biases may shift absolute refusal estimates despite comparable paired within-model contrasts.

Ethics Statement

The study uses published or publicly derived harmful content, controlled image-query conditions, and programmatic or judge-model evaluation. Its materials and procedures are documented across the appendix.

  • Harmful queries and images come from published safety benchmarks or their public intents and introduce no novel attack vectors.The work studies VLM safety mechanisms, while ablation degrades safety only in controlled settings.
  • The appendix documents the neuron pipeline, dataset construction, evaluation protocol, and extended behavioral and mechanistic analyses.
  • The benchmark retains 2,417 harmful samples and a size-matched safe set after harm and applicability scoring, with near-duplicates removed.
  • The evaluation spans 472 intents across 9 categories, 8 image conditions, and 3 query types, producing approximately 11,300 samples per model.
  • VISAFE-DETECT pairs harmful and safe images with a shared neutral procedural query to isolate visual effects from textual confounds.Harmful images are drawn from five categories, while safe images come from COCO and Flickr30K.
  • Refusal judgments use a fixed GPT-4o prompt, while safe queries use a separate over-refusal prompt.

D.1 Full Factorial Results

Across the full factorial and ablation analyses, image conditions produce model-specific refusal shifts, while text-neuron ablation changes refusal more consistently than visual-neuron ablation. Decision-versus-phrasing effects vary across models, with limited changes in general capabilities.

  • Full factorial results: Image substitutions produce model-specific shifts in direct-harmful refusal, without a consistent cross-model ordering.The analysis covers all eight image conditions.
  • Full factorial results: Rendered-text images trigger 10–60% over-refusal, whereas most models show less than 5% on text-only inputs.This pattern is consistent with OCR-mediated access to rendered content.
  • Phrasing versus decision separation: The RR-Judge/RR-KW ratio spans 0.09 to 1.88 on ViSafe-472, indicating a cross-model gradient between phrasing-level and decision-level effects.The ratio depends on both model and query distribution.
  • Phrasing versus decision separation: 8 of 10 models show decision-level transfer, with ratios from 0.37 to 1.88; the two lowest-ratio models primarily lose canonical refusal templates.Main ablation results use RR-Judge, and the text–visual asymmetry also holds within models showing decision-level transfer.
  • Capability preservation: Safety-neuron ablation changes MMLU accuracy by no more than 0.3 points and MMMU accuracy by within ±1.1% in tested models.The capability evaluations compare clean, text-neuron-ablation, and visual-neuron-ablation conditions.
  • Component analysis: Visual safety neurons span four components; in the reported breakdown, mlp_out contributes 6 of 40 neurons and self_attn contributes 5.The component analysis distinguishes visual scanning from text scanning, which omits mlp_out.

G.1 GEVD Full Results

GEVD scaling compares how safety effects change when projecting out 5 versus 50 directions across models. Text effects remain nearly flat beyond five directions, whereas visual effects continue growing through fifty, with a random-direction null showing the same contrast on Qwen2.5-VL.

  • Multi-model scaling: Across four models with measurable visual signals, text effects change by 1–5 points from k=5 to k=50, while visual effects grow by 9–35 points.GLM-4.1V-9B-Thinking is excluded from the visual analysis because keyword-rated baseline refusal is below 5%.
  • Multi-model scaling: Text changes little beyond k=5, but visual effects continue growing through k=50 across the GEVD model results.The table reports keyword-rated refusal changes at the best layer.
  • Random-direction null: On Qwen2.5-VL, random-direction null adjustment leaves the text effect flat while the visual effect keeps growing.The comparison is available for the model shown in the main-text Tab. 6 analysis.
Loading 2608.30750v1…