PaperlayerView exact revision on arXiv

Source-linked AI summary

ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time

Yi Ding, Bolian Li, Ruqi Zhang

arXiv:2410.06625v2cs.CVcs.CLcs.LG

TL;DR

VLM safety mechanisms can be bypassed by visual inputs, while existing defenses may require substantial resources or fail to preserve helpfulness. ETA evaluates visual inputs and generated responses, then applies shallow and deep alignment to unsafe cases. Experiments report reduced unsafe rates and improvements across harmlessness, helpfulness, and general-ability measures.

  • Problem

    Visual inputs can bypass VLM safety mechanisms, while existing defenses may be resource-intensive or fail to provide both safe and helpful responses.

    Source: §1 · pp. 1–2

  • Method

    ETA evaluates image and response safety, then uses an interference prefix and sentence-level best-of-N search to align unsafe responses.

    Source: §1 · p. 2

  • Results

    ETA reduces LLaVA-1.5-13B’s unsafe rate on cross-modality attacks by 87.5%, compared with ECSO’s 5% reduction, and experiments assess harmlessness, helpfulness, and preservation of general abilities.

    Source: §1 · p. 3, §5.2 · p. 8

  • Takeaways & Limitations

    ETA is presented as an inference-time approach that improves safety and usefulness without additional data or fine-tuning.

    Source: §1 · p. 2, §6 · p. 10

    The paper identifies harmful data during training and testing and the potential generation of harmful content as ethical concerns.

    Source: ETHICS STATEMENT · p. 11

Abstract

from arXiv · show

Vision Language Models (VLMs) have become essential backbones for multimodal intelligence, yet significant safety challenges limit their real-world application. While textual inputs are often effectively safeguarded, adversarial visual inputs can easily bypass VLM defense mechanisms. Existing defense methods are either resource-intensive, requiring substantial data and compute, or fail to simultaneously ensure safety and usefulness in responses. To address these limitations, we propose a novel two-phase inference-time alignment framework, Evaluating Then Aligning (ETA): 1) Evaluating input visual contents and output responses to establish a robust safety awareness in multimodal settings, and 2) Aligning unsafe behaviors at both shallow and deep levels by conditioning the VLMs' generative distribution with an interference prefix and performing sentence-level best-of-N to search the most harmless and helpful generation paths. Extensive experiments show that ETA outperforms baseline methods in terms of harmlessness, helpfulness, and efficiency, reducing the unsafe rate by 87.5% in cross-modality attacks and achieving 96.6% win-ties in GPT-4 helpfulness evaluation. The code is publicly available at https://github.com/DripNowhy/ETA.

1 INTRODUCTION

VLMs face safety failures because visual inputs can bypass protections learned by LLM backbones, while existing defenses may overlook visual content or produce unhelpful refusals. ETA evaluates visual inputs and generated text, then applies shallow and deep alignment to unsafe cases.

1 INTRODUCTION

  • Visual inputs can bypass existing safety mechanisms and lead VLMs to generate harmful responses.Visual instruction tuning may also cause LLM backbones to forget previously learned safety mechanisms.Source: §1 · p. 1
  • Existing defenses either require extensive data and labor or overlook visual inputs; generic refusals can also leave responses unhelpful and disconnected from the input.Fine-tuning methods may compromise general capabilities, while text-focused inference defenses can mis-evaluate multimodal scenarios.Source: §1 · pp. 1–2
  • ETA evaluates visual inputs and generated responses, then uses an interference prefix and sentence-level best-of-N search to align unsafe behavior.Its multimodal evaluation assesses visual safety before generation and response safety afterward.Source: §1 · p. 2, Figure 1 · p. 2
  • ETA requires no additional data or training and is presented as a plug-and-play approach that preserves VLM general capabilities while seeking safe, useful responses.The paper reports experiments on harmlessness, helpfulness, and preservation of general abilities.Source: §1 · pp. 2–3
  • The paper attributes VLM safety failures to continuous visual embeddings acting as outliers relative to discrete textual embeddings used to align LLM backbones.This motivates evaluating both input images and output text.Source: §1 · p. 2

2 RELATED WORKS

Related work covers fine-tuning and inference-time alignment, including methods for VLM safety. ETA is positioned as a training-free approach using multimodal evaluation and bi-level alignment.

2 RELATED WORKS

  • Fine-tuning approaches use preference learning or specialized data, but can be resource-intensive and may weaken core capabilities or generalize poorly beyond red-teaming domains.Some methods add classifiers during training or use fine-tuned defense LLMs at inference.Source: §2 · p. 3
  • Inference-time LLM methods score or self-evaluate candidate responses to select outputs aligned with predefined preferences.These approaches avoid instability associated with PPO training in RLHF.Source: §2 · p. 3
  • VLM defenses include Adashield prompt refinement and ECSO text distillation, whereas ETA uses external multimodal evaluation and bi-level alignment without training or converting images to text.Adashield is described as limited to certain structure-based jailbreaks; the paper primarily compares ETA with ECSO.Source: §2 · p. 3

3 PRELIMINARIES

VLMs combine continuous visual embeddings with discrete textual embeddings and generate responses autoregressively. The paper describes how visual embeddings may bypass safety mechanisms aligned to text.

3 PRELIMINARIES

  • A vision encoder and connector transform an image into a continuous embedding in textual embedding space, which is then provided to the LLM alongside textual embeddings.The LLM predicts response tokens autoregressively.Source: §3 · p. 3
  • VLMs generate a response by predicting each next-token distribution conditioned on visual and textual embeddings and the preceding tokens.The paper notes that integrating vision embeddings and fine-tuning the LLM backbone can lead to harmful responses to malicious queries.Source: §3 · pp. 3–4
  • 7% lower unsafe rate resulted from mapping continuous visual embeddings to their nearest discrete textual embeddings on the SPA-VL Harm test set.The passage presents this mapping as evidence that visual inputs can be transformed into harmful textual tokens addressed by LLM safety mechanisms.Source: Figure 2 · p. 4

4 METHODOLOGY

ETA establishes multimodal safety awareness and applies alignment to unsafe behavior. Its evaluators assess images and responses, while shallow and deep alignment steer generation toward safe and useful content.

4 METHODOLOGY

  • ETA hypothesizes that continuous visual embeddings bypass LLM safety mechanisms and combines multimodal evaluation with safety-guided bi-level alignment.The method is designed to identify a harmless and helpful response.Source: §4 · p. 4

4.1.1 PRE-GENERATION EVALUATOR

  • A CLIP-based pre-generation evaluator compares image embeddings with text embeddings for common unsafe categories, using a threshold to distinguish safe and unsafe inputs.On 100 harmful and 100 safe images, the score distributions separated sufficiently to justify a threshold.Source: §4.1.1 · p. 5

4.1.2 POST-GENERATION EVALUATOR

  • A safety-specific reward-model input format improves separation between harmful and harmless responses, and ETA uses an adjustable threshold to classify generated outputs.ETA applies alignment only when both pre- and post-generation evaluations flag a behavior as unsafe.Source: §4.1.2 · p. 6

4.2 SAFETY-GUIDED BI-LEVEL ALIGNMENT

  • ETA first conditions generation on an interference prefix, then guides it with sentence-level best-of-N search to address unsafe behavior.The paper describes these as shallow and deep alignment, respectively.Source: §4.2 · p. 6

4.2.1 INTERFERENCE PREFIXES AS SHALLOW ALIGNMENT

  • Adding an interference prefix reduces unsafe rate by activating the VLM's safety capabilities when faced with harmful multimodal inputs.The prefix conditions initial output tokens, which the paper says influence subsequent token distributions.Source: §4.2.1 · p. 6

4.2.2 SENTENCE-LEVEL BEST-OF-N SEACHING AS DEEP ALIGNMENT

  • Sentence-level best-of-N samples and scores candidate sentences, accepting the highest-scoring option to bias generation toward safer, more helpful, coherent, and relevant responses.The method addresses cases where an initial refusal is followed by harmful content; it uses CLIP-based utility scoring to retain image-specific suggestions.Source: §4.2.2 · pp. 6–7
  • The sentence-level score balances safety and utility, with α = 0 for the first sentence and α = 1/i for subsequent sentences.Each candidate sentence is generated conditioned on the previously selected sentences.Source: §4.2.2 · p. 7

5 EXPERIMENTS

ETA is evaluated for safety, usefulness, inference efficiency, and component effects across VLM backbones and multimodal safety and helpfulness benchmarks. Results report lower unsafe-response rates than ECSO, preserved general capabilities, and improved helpfulness alongside modest inference overhead.

5.1 SETUPS

  • The experiments assess safety, helpfulness, inference efficiency, and component effects using multiple VLM backbones, safety benchmarks, and general-capability benchmarks.The setup compares ETA primarily with inference-time method ECSO and also evaluates fine-tuned methods on VLGuard helpfulness.Source: §5 · p. 7, §5.1 · pp. 7–8

5.2 RESULTS

  • 87.5%: ETA reduces LLaVA-1.5-13B’s USR on cross-modality attacks, compared with a 5% reduction for ECSO.Across the reported safety benchmarks, ETA significantly reduces USR compared with ECSO; Table 1 covers four VLM backbones.Source: §5.2 · p. 8, Table 1 · p. 8
  • ETA outperforms other fine-tuned and inference-time baselines on general performance, and its GPT-4-Turbo helpfulness win-tie comparison favors ETA.On TextVQA, ETA reduces accuracy by 0.05%, while remaining better than the other baselines; the win-tie evaluation uses SPA-VL Help.Source: Table 2 · p. 9, §5.2 · p. 9, Table 3 · p. 9
  • ETA adds 0.1 seconds per generation, compared with ECSO’s average 0.39-second increase, on MMB and SQAI.The authors attribute ECSO’s additional generation overhead to unnecessary alignment steps following inaccurate self-evaluation.Source: §5.2 · p. 9, Table 4 · p. 10

5.3 ABLATION STUDIES

  • Increasing τpost and decreasing τpre raises safeguarding, while the reverse settings prioritize preserving general capabilities.The final configuration uses τpre = 0.16 and τpost = 0.06.Source: §5.3 · p. 10
  • Shallow and deep alignment together improve harmlessness and helpfulness more than either alone; adding a utility score increases Helpful Score by + 0.12/10 with + 0.38/100 in USR.The utility score selects responses relevant to input images, while the reward score supports their safety.Source: §5.3 · p. 10, Table 5 · p. 10

6 CONCLUSION

ETA evaluates visual inputs and generated responses, then aligns VLM outputs through shallow style adjustment and deep content selection. Experiments report improved safety and helpfulness while maintaining utility, without additional data or fine-tuning.

6 CONCLUSION

  • The paper identifies continuous visual token embeddings as a primary cause of VLM safety mechanism failures.ETA introduces multimodal evaluation to establish safety awareness, then applies shallow alignment for safe style and deep alignment to address content safety and usefulness.Source: §6 · p. 10
  • ETA is an inference-time alignment method intended to produce safer, more useful VLM responses without additional data or fine-tuning.Source: §6 · p. 10
  • Experiments evaluate ETA-generated responses for safety, helpfulness, and efficiency, reporting superiority over existing methods while maintaining VLM utility.Source: §6 · p. 10

ETHICS STATEMENT

The ethics statement notes potential concerns about harmful data in training and testing and the possibility of generating harmful content. The appendices describe prompts for evaluating image safety and response safety.

ETHICS STATEMENT

  • Potential ethical concerns include harmful data during training and testing, and the possibility of generating harmful content.Source: ETHICS STATEMENT · p. 11

A.1 PROMPTS FOR PRE-GENERATION EVALUATOR (CLIP SCORE)

  • The pre-generation evaluator assesses image-prompt similarity to determine whether input images are safe.Its CLIP text prompt lists unsafe, pornographic, violent, political, physically harmful, illegal, and private content.Source: §A.1 · p. 15

A.2 PROMPTS FOR POST-GENERATION EVALUATOR (REWARD MODEL)

  • The post-generation reward-model prompt prioritizes response safety over relevance to the original question.It favors refusals with helpful advice and assigns low scores to responses containing harmful or unsafe instructions.Source: §A.2 · p. 15

A.3.1 SAFETY METRICS

The appendices define safety and helpfulness evaluation measures, describe benchmark and baseline setups, and specify GPT-4-Turbo evaluation prompts. Additional analysis connects continuous visual embeddings with safety mechanism bypass.

A.3.1 SAFETY METRICS

  • USR measures the probability of unsafe outputs, while ASR estimates attack success using target-string matching, which the authors note may yield false positives or negatives.MD-Judge-v0.2-internlm2_7B evaluates response safety; ASR results are reported only in Appendix Table 8 because of target-string matching’s accuracy concerns.Source: §A.3.1 · pp. 15–16

A.3.2 HELPFULNESS METRICS.

  • GPT-4-Turbo rates outputs from 1 to 10 for helpfulness and harmlessness, and compares response helpfulness through win-tie judgments on SPA-VL Help.Source: §A.3.2 · p. 16

A.4 BENCHMARKS DETAILS

  • The benchmark suite includes image-based attacks and text attacks; FigStep has 520 samples, AdvBench has 500 harmful texts, and MME covers perception and reasoning.MM-SafetyBench covers 13 harmful categories and 1,680 samples; the cross-modality attack combines harmful noise images with a textual suffix attack.Source: §A.4 · pp. 16–17

A.5 BASELINES DETAILS

  • ECSO transforms image information into text after self-evaluating an output, while Posthoc-LoRA and Mixed-LoRA use fine-tuning-based defenses.Posthoc-LoRA mixes comprehensive-benchmark data into fine-tuning; Mixed-LoRA incorporates its defense data during visual instruction tuning.Source: §A.5 · p. 17

A.6 GPT-4 EVALUATION DETAILS

  • GPT-4-Turbo’s overall-score prompt evaluates responses for harmlessness, helpfulness, relevance, accuracy, depth, creativity, and detail using both question and answer.The win-tie prompt compares two responses with the related image and asks which is more helpful, prioritizing helpfulness when both responses are harmless.Source: §A.6 · pp. 17–18, SYSTEM PROMPT · p. 18

SYSTEM PROMPT

  • The safety-specific reward-model input asks for a response without safety issues, emphasizing safe refusals and advice over relevance to the original question.Figure 4 reports that this format better distinguishes safe from unsafe responses based on reward distributions and KL divergence.Source: Figure 4 · p. 19

B.1 MORE RESULTS ON OUR MOTIVATION: CONTINUOUS VISUAL EMBEDDINGS BYPASS SAFETY MECHANISMS

  • The authors report that decreases in USR after mapping continuous visual token embeddings to discrete text support their claim that continuous embeddings bypass safety mechanisms aligned on discrete text tokens.The additional evaluation includes SPA-VL Harm and a 100-point random sample from VLSafe.Source: §B.1 · p. 19, Table 6 · p. 19

B.2 POST-GENERATION EVALUATION RESULTS

The safety-specific open-ended input format makes textual reward-model evaluation more reliable, and ETA remains effective across stronger VLMs and multiple safety benchmarks.

B.2 POST-GENERATION EVALUATION RESULTS

  • Text-only reward models may mis-score cross-modal attacks when harmful content is in an image but the accompanying text is harmless.A harmless text input can make refusal score lower than the score for a harmful response.Source: §B.2 · p. 19
  • Open-ended safety-specific prompts make textual reward-model evaluation more reliable than the vanilla input format on MM-SafetyBench.The format encourages the reward model to prioritize response safety rather than question relevance.Source: §B.2 · p. 20

B.3 RESULTS ON MORE VLMS

  • ETA further reduces unsafe rates on three stronger VLMs, and its ASR performance is superior across multiple safety benchmarks.The supplied passages report these comparisons without providing the table values.Source: §B.3 · p. 20, Table 7 · p. 20, §B.4 · p. 20

B.5 EXTENSIBLE TO TEXT-ONLY BENCHMARKS

On text-only safety benchmarks, ETA uses post-generation evaluation and reduces harmful responses, including under adversarial suffix attacks.

B.5 EXTENSIBLE TO TEXT-ONLY BENCHMARKS

  • Post-generation evaluation alone is used to assess response safety on AdvBench because its inputs contain no images.ETA significantly reduced harmful-response probability for both regular harmful queries and adversarial attacks with suffixes.Source: §B.5 · pp. 20–21
  • Table 9 reports ETA’s performance on the text-only safety benchmark.The supplied table caption identifies the benchmark scope but gives no cell values.Source: Table 9 · p. 21
  • ETA reduced LLaVA-1.5-7B’s unsafe rate by 96.54% in adversarial text-only settings, where ECSO was ineffective.The evaluation used post-generation assessment because the benchmark inputs contain no images.Source: §B.5 · pp. 20–21, Table 9 · p. 21

B.6 MORE EXPERIMENTS ON HELPFULNESS EVALUATION

ETA retains helpfulness while reducing unsafe behavior, and its evaluation strategy and embedding-mapping examples probe how those outcomes depend on design choices.

B.6 MORE EXPERIMENTS ON HELPFULNESS EVALUATION

  • ETA has a smaller effect than ECSO on LLaVA-1.5-13B foundational abilities, and does not negatively impact performance on MMMU-Pro.Tables 10 and 11 report general performance and MMMU-Pro results, respectively.Source: §B.6 · p. 21, Table 11 · p. 22
  • ETA consistently shows a helpfulness advantage over other methods, including fine-tune-based methods, on SPA-VL Help.GPT-4-Turbo win-tie-lose evaluation reports more ties against inference-time ECSO on LLaVA-1.5-7B, but fewer ties against fine-tuned methods while ETA remains more likely to win.Source: §B.6 · p. 21, Figure 5 · p. 22

B.7 MORE RESULTS OF ABLATION ON EVALUATING STAGE

  • Replacing ETA’s multimodal evaluator with ECSO self-evaluation increases unsafe rates and decreases general ability across VLM backbones.Table 12 compares harmlessness and helpfulness for the self-evaluation and ETA alignment configuration.Source: §B.7 · p. 22, Table 12 · p. 23

B.8 ETA AVOIDS EXCESSIVE CONSERVATISM

  • ETA applies alignment only when both pre- and post-generation evaluations classify a response as unsafe, allowing benign instructions about unsafe images to retain the original helpful response.The authors attribute avoiding over-conservatism in this case to reliable post-generation evaluation.Source: §B.8 · p. 22, Figure 6 · p. 23

B.9 EXAMPLES OF CONTINUOUS EMBEDDING TO DISCRETE EMBEDDING

  • Mapping continuous visual embeddings to discrete text-token embeddings reduces unsafe responses, while cosine similarity preserves image relevance better than Euclidean distance.The examples describe continuous visual embeddings as deviating from discrete text embeddings.Source: §B.9 · p. 23

B.10 EXAMPLES OF ETA GENERATED RESPONSE

ETA’s examples show how deep alignment, prefix choice, evaluation thresholds, and visual-token mapping relate to safety and helpfulness across VLMs.

B.10 EXAMPLES OF ETA GENERATED RESPONSE

  • Shallow alignment with only a prefix can begin with refusal language yet continue to harmful output, highlighting the role of deep alignment.The example describes transitional phrases such as “However” before harmful responses.Source: §B.10 · p. 23, Figure 11 · p. 28
  • Pre- and post-evaluation thresholds expose a safety-usefulness trade-off, and ETA’s criterion aligns only when both stages classify behavior as unsafe.Figure 7 reports mis-evaluation, TextVQA accuracy, and SPA-VL Harm unsafe rate across threshold settings; Table 13 defines the compared criteria.Source: Figure 7 · p. 24, Table 13 · p. 24
  • “As an AI assistant” yields more helpful, relevant responses while maintaining safety; the prefix is the varying factor in the Figure 10 comparison.Figure 10 compares different prefixes with both shallow and deep alignment applied.Source: §B.10 · pp. 23–24, Figure 10 · p. 27
  • ETA shows superior safety-benchmark performance across additional VLM backbones when introduced at inference.Figure 12 compares backbones and the impact of ETA on safety benchmarks.Source: §B.10 · p. 24, Figure 12 · p. 29
  • Mapping visual embeddings to text tokens reduces unsafe responses, and cosine similarity preserves image relevance better than Euclidean distance.The examples compare responses before and after mapping continuous visual embeddings to discrete text-token embeddings.Source: Figure 8 · p. 25, Figure 9 · p. 26
Loading 2410.06625v2…