PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models

Shicheng Xu, Liang Pang, Yunchang Zhu, Huawei Shen, Xueqi Cheng

arXiv:2410.12662v2cs.CVcs.AIcs.CL

TL;DR

LVLMs handle toxic images less safely than toxic text, and existing alignment methods do not explain or accomplish transfer of text safety to vision. The paper traces this to hidden-state misalignment and proposes TGA, which uses retrieved text to guide image representations. TGA transfers text safety without visual safety fine-tuning while maintaining general vision-task performance; the reported failure scenarios include inconspicuous harmful objects, bloody images, and images the LVLM cannot understand well.

  • Problem

    Existing vision-language alignment fails to explain or effectively transfer LLM text safety to vision, leaving LVLMs more vulnerable to toxic images than toxic text.

    Source: §1 · pp. 1–2

  • Method

    TGA retrieves text related to an image and uses it to guide the image projection into the LLM hidden-state space.

    Source: §1 · p. 2

  • Results

    TGA transfers the basic LLM’s text safety mechanism to vision without visual safety fine-tuning and maintains general performance across vision tasks.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    The paper attributes failed safety transfer to insufficient hidden-state alignment that shifts image semantics and misleads safety layers.

    Source: §1 · p. 2

    Reported failure scenarios center on inconspicuous harmful objects, bloody-domain images, and images the LVLM cannot correctly caption.

    Source: §A.4 · p. 15

Abstract

from arXiv · show

Vision-language alignment in Large Vision-Language Models (LVLMs) successfully enables LLMs to understand visual input. However, we find that existing vision-language alignment methods fail to transfer the existing safety mechanism for text in LLMs to vision, which leads to vulnerabilities in toxic image. To explore the cause of this problem, we give the insightful explanation of where and how the safety mechanism of LVLMs operates and conduct comparative analysis between text and vision. We find that the hidden states at the specific transformer layers play a crucial role in the successful activation of safety mechanism, while the vision-language alignment at hidden states level in current methods is insufficient. This results in a semantic shift for input images compared to text in hidden states, therefore misleads the safety mechanism. To address this, we propose a novel Text-Guided vision-language Alignment method (TGA) for LVLMs. TGA retrieves the texts related to input vision and uses them to guide the projection of vision into the hidden states space in LLMs. Experiments show that TGA not only successfully transfers the safety mechanism for text in basic LLMs to vision in vision-language alignment for LVLMs without any safety fine-tuning on the visual modality but also maintains the general performance on various vision tasks (Safe and Good).

1 INTRODUCTION

LVLMs inherit text safety mechanisms, yet current vision-language alignment does not effectively transfer them to vision, leaving toxic images harder to handle than toxic text. The paper links this gap to hidden-state semantic misalignment and proposes TGA to guide image representations with retrieved text.

  • Current alignment methods fail to transfer text safety effectively, and LVLMs handle same-semantic toxicity differently across text and vision.Source: §1 · pp. 1–2
  • Specific local transformer layers’ hidden states are crucial to safety activation, but insufficient vision-language alignment shifts image semantics and misleads those layers.Source: §1 · p. 2, Figure 1 · p. 2
  • TGA uses retrieved text related to an input image to guide its projection into the LLM hidden-state space.Source: §1 · p. 2
  • TGA transfers the basic LLM’s text safety mechanism to vision without visual safety fine-tuning while maintaining general performance across vision tasks.Source: §1 · p. 2

2 RELATED WORK

Prior work equips LLMs to process visual input through vision-language alignment, but whether this extends all text capabilities to vision remains unknown. LVLM safety studies identify vulnerability and pursue safety tuning, while leaving the transfer explanation and data-collection burden unresolved.

  • Vision-language alignment equips basic LLMs to process visual input through training on large-scale text-image pairs, as in LLaVA, InstructBLip, Qwen-VL, MiniGPT-4, Flamingo, and PaLM-E.Source: §2 · p. 2
  • Whether vision-language alignment extends all LLM capabilities from text to vision remains unknown, and this paper identifies safety as a capability not transferred by typical methods.Source: §2 · p. 3
  • Prior LVLM safety work reports toxic-output vulnerability and explores safety instruction-tuning, which requires more multimodal data and human effort than text-only tuning.Source: §2 · p. 3

3 SAFETY MECHANISM CANNOT BE CROSS-MODAL TRANSFERRED

The experiments compare safety responses to toxic inputs with matching semantics across text and vision. They find that text safety is largely preserved in LVLMs, but is not effectively transferred to vision.

  • The assessment uses toxic text-image pairs with matched semantics, constructed by captioning images from HOD and ToViLaG datasets.Source: §3 · p. 3
  • The evaluation measures whether models refuse toxic responses using Defence Success Rates, with LLaMA-2-7B judging response toxicity.Source: §3 · p. 3
  • Across tested models and toxic scenes, LVLM and basic-LLM DSRs are close on toxic text, while LVLMs struggle to defend against same-semantic toxic images.Source: §3 · p. 3

4 CAUSE OF FAILURE IN CROSS-MODAL TRANSFERRING SAFETY

Safety activation depends on hidden states at specific transformer layers, but existing vision-language alignment does not align image and text states sufficiently at those layers. This mismatch impairs image-to-text safety transfer, while reconstruction experiments support the importance of hidden-state alignment.

4.1 SAFETY MECHANISM IS ACTIVATED AT SPECIFIC LAYERS BY HIDDEN STATES

  • Safety activation coincides with peaks in attention to toxic-token hidden states, and masking information from those layers disrupts safety most.Activation layers are located by detecting when apology-related tokens begin ranking first in changes to the next-token distribution.Source: §4.1 · pp. 4–6

4.2 INSUFFICIENT ALIGNMENT AT HIDDEN STATES MISLEADS SAFETY MECHANISM

  • At safety-activation layers, text-image hidden-state similarity is lower than CLIP semantic similarity, indicating insufficient hidden-state alignment.The comparison uses hidden states for text and image inputs with the same semantics.Source: §4.2 · p. 6, Figure 4 · p. 6
  • Adding mean-pooled text hidden states to image states improves toxic-image safety, especially when applied at identified safety-activation layers.The intervention tests hidden states across layers by adding text states before the next transformer layer.Source: Figure 5 · p. 6, §4.2 · p. 7
  • Existing vision-language methods insufficiently align image and text hidden states at safety-activation layers, so those layers cannot correctly assess image toxicity.The paper links this mismatch to the collapse of the safety mechanism on vision.Source: §4 · p. 3, §4.2 · p. 7

5 TEXT GUIDED ALIGNMENT AT HIDDEN STATES LEVEL

TGA addresses insufficient hidden-state alignment between vision and language, which the paper identifies as a critical reason safety mechanisms fail to transfer. It uses text related to images to guide alignment and reports improved toxic-image safety without visual safety fine-tuning while maintaining performance on vision tasks.

5 TEXT GUIDED ALIGNMENT AT HIDDEN STATES LEVEL

  • Insufficient hidden-state alignment in safety-activation layers is identified as a critical reason safety mechanisms collapse when transferred from text to vision.TGA is proposed to align vision and language at the hidden-state level.Source: §5 · p. 7

5.1 MOTIVATION AND OVERVIEW

  • Prior methods align text output to visual input, whereas TGA uses retrieved semantically relevant text to guide vision-to-language alignment between hidden states.The paper characterizes the prior input-to-output approach as asymmetrical and says it neglects internal visual representations.Source: §5.1 · p. 7

5.2.1 DATA CONSTRUCTION

  • For each training image, TGA retrieves a relevant text and generates a textual caption to represent the image’s semantics in text modality.The retrieval corpus contains 1,153K image captions; the provided data-construction passage describes it as having little toxicity.Source: §5.2.1 · p. 8

5.2.2 TRAINING

  • TGA uses caption hidden states as targets and retrieved text to guide image hidden states toward them at each transformer layer.Its pair-wise loss aims for image states to be closer to caption states than retrieved-text states, and the total loss combines this guidance with language-modeling cross-entropy.Source: §5.2.2 · p. 8

5.3.1 EXPERIMENTAL DETAILS

  • Evaluation tests toxic-image defence without visual safety fine-tuning, using Defence Success Rates to measure refusal of toxic responses.The test set contains 20,531 toxic images across seven categories; mainstream alignment methods and Unlearn-FigS are included as baselines.Source: §5.3.1 · p. 9

5.3.2 EXPERIMENTAL RESULTS

  • TGA significantly improves toxic-image safety over mainstream alignment methods and outperforms Unlearn-FigS, without additional visual safety fine-tuning.The reported safety comparison covers seven scenes; the supplied table caption does not provide cell values.Source: §5.3.2 · p. 9, Table 2 · p. 9
  • TGA maintains comparable performance on vision tasks; its retrieved-text guidance improves safety and general performance, while inference-time retrieval alone does not significantly improve baseline safety.The comparison of general vision-task performance is presented in Table 3.Source: §5.3.2 · pp. 9–10, Table 3 · p. 10, Figure 7 · p. 10

6 CONCLUSION AND DISCUSSION

The paper frames toxic-image vulnerability as a cross-modal safety-transfer problem, tracing it to insufficient hidden-state alignment. It proposes an alignment method intended to transfer text safety to vision while maintaining general vision-task performance.

6 CONCLUSION AND DISCUSSION

  • The paper introduces Cross-Modal Safety Mechanism Transfer as a perspective for explaining and addressing LVLMs’ greater vulnerability to toxic vision than toxic text.Source: §6 · p. 10
  • The authors attribute failed transfer to insufficient vision-language alignment at the hidden-state level.Source: §6 · p. 10
  • Their alignment method transfers basic LLM text-safety mechanisms to vision while maintaining general performance across vision tasks compared with existing SoTA LVLMs.Source: §6 · p. 10

A.1 GENERAL CAPABILITIES OF LVLMS ON VISION TASKS

This section describes benchmarks and metrics used to assess LVLM general capabilities across scientific question answering, hallucination, image and video comprehension, and visual conversation.

A.1 GENERAL CAPABILITIES OF LVLMS ON VISION TASKS

  • ScienceQA evaluates zero-shot generalization on its image subset for scientific question answering using accuracy.It contains 21k multimodal multiple-choice questions with science topics and answer annotations, lectures, and explanations.Source: §A.1 · p. 13
  • POPE measures hallucination on random, common, and adversarial COCO subsets using F1 score.Source: §A.1 · p. 13
  • SEED-Bench assesses image, video, and overall performance across 12 dimensions, reporting accuracy.It contains 19K multiple-choice questions with human annotations.Source: §A.1 · p. 13
  • MM-Vet evaluates visual-conversation responses across diverse tasks for correctness and helpfulness using GPT-4 evaluation.Source: §A.1 · p. 13

A.2 ADDITIONAL EXPERIMENTAL RESULTS

The additional experiments assess TGA’s robustness to caption errors, unsafe compositional inputs, and jailbreak attacks, and compare it with other defenses and a safety-aligned LLM.

Robustness to Caption Errors

  • TGA performs relatively well when caption-noise ratios are below 10% in a robustness test using a randomly selected 100k-sample subset.Captions were perturbed by replacement or random information deletion at ratios of 5%, 10%, 15%, and 20%; the authors describe robustness as acceptable.Source: §A.2 · p. 14

Robustness to Unsafe Compositional Inputs

  • TGA achieves state-of-the-art performance on SIUO unsafe compositional inputs, compared only with LLaVA-1.5 under matched training data.The authors explain that hidden-state alignment lets vision and language share a safety mechanism for combined-input attacks; Qwen-VL is excluded because it uses much more and partly in-house training data.Source: §A.2 · p. 14, Table 5 · p. 14

Robustness to Jailbreak Attacks

  • TGA is more robust than baselines against role-play, in-context-learning, and FigStep visual-prompt jailbreak attacks, measured by Defence Success Rates.Source: §A.2 · p. 14, Table 6 · p. 14

Comparison with More Defense Methods

  • TGA outperforms Tovilag and ECSO in the reported comparison.The authors contrast Tovilag’s additional detoxification fine-tuning with ECSO’s post-hoc safety assessment of responses.Source: §A.2 · pp. 14–15, Table 7 · p. 15

Comparison with Upper Bound Estimate

  • The upper-bound experiment trains a ViT-based filter on mixed toxic and normal images to filter toxic images directly.The supplied passage identifies this direct image filter as an upper bound but gives no result value.Source: §A.2 · p. 15, Table 8 · p. 15

How about Safety-aligned LLMs?

  • Even when initialized from a language-safety-aligned LLM, the resulting LVLM does not effectively transfer that safety mechanism to vision.Source: §A.2 · p. 15, Table 9 · p. 15

A.3 CASE STUDY

The section presents case studies of responses from different LVLMs to toxic images.

A.3 CASE STUDY

  • Figures 8–14 present case studies of different LVLMs’ responses to toxic images.Source: §A.3 · p. 15

A.4 DISCUSSION ON BAD CASES

The discussion identifies three bad-case classes: harmful objects that are not easily noticed, bloody-domain images, and images that are hard for the LVLM to understand, including images it cannot correctly caption.

A.4 DISCUSSION ON BAD CASES

Loading 2410.12662v2…