PaperlayerView exact revision on arXiv

Source-linked AI summary

Answer It or Not? Towards Prompt-Adapted Adversarial Images on Vision Language Models

Shunyuan Bai, Yuchen Ren, Le Yang, Chenhao Lin, Chao Shen, Zhengyu Zhao

arXiv:2610.16622v1cs.CVcs.AI

TL;DR

Existing VLM adversaries largely overlook prompt dynamics, while prompt-agnostic protections reject legitimate as well as harmful questions. ProAda jointly optimizes visual perturbations for harmful and general prompts and updates harmful-prompt embeddings under a harmfulness check. On visual-privacy-related images, it rejects over 90% of mostly unseen harmful prompts while maintaining correct answers to over 90% of mostly unseen general prompts, outperforming prompt-agnostic VLM adversaries overall.

  • Problem

    Existing VLM adversaries are prompt-agnostic, although visual privacy protection calls for adapting adversarial effects to prompt properties.

    Source: §1 · p. 2

  • Method

    ProAda jointly optimizes visual perturbations for harmful and general prompts and iteratively updates harmful-prompt embeddings using a classifier to check that they remain harmful.

    Source: §1 · p. 2

  • Results

    ProAda rejects over 90% of mostly unseen harmful prompts while maintaining correct answers to over 90% of mostly unseen general prompts, outperforming prompt-agnostic VLM adversaries overall.

    Source: §1 · p. 2

  • Takeaways & Limitations

    In its visual-privacy case study, ProAda balances privacy protection with general image utility.

    Source: §1 · p. 2

    Prompt adaptation is limited to harmful-or-general binary options and visual privacy protection, and cross-model transfer remains limited, especially in targeted settings.

    Source: Limitations · p. 9

Abstract

from arXiv · show

With the advancement in language modeling, research on adversarial images has recently shifted from image classifiers to vision-language models (VLMs). Despite the new, text-prompt-driven nature of VLMs, we find that existing VLM adversaries have largely overlooked the dynamics of text prompts. In a typical use case, adversarial images can protect visual privacy by rejecting VLMs from answering questions about the image. However, existing protections are prompt-agnostic, i.e., indiscriminately rejecting any question, including those for potential legitimate use of the image. To address this limitation, we propose Prompt-Adapted (ProAda) adversarial images, which are optimized to balance the rejection of harmful prompts and the correct response of legitimate prompts. In particular, to achieve a more accurate adversarial direction, the text prompt is also iteratively updated with a simple classifier judging whether the harmfulness of the prompt remains. Extensive experiments on visual-privacy-related datasets validate that ProAda can reject over 90% of harmful prompts while allowing over 90% of legitimate ones, demonstrating superior overall performance compared to existing prompt-agnostic VLM adversaries. In addition, our ProAda achieves competitive overall cross-model transferability and improved robustness against countermeasures.

1 Introduction

Existing VLM adversaries overlook prompt dynamics, although visual privacy protection must distinguish harmful prompts from legitimate ones. ProAda adapts adversarial effects to prompts while balancing privacy protection and general image utility.

1 Introduction

  • Existing VLM adversaries are prompt-agnostic, disrupting either one prompt or all prompts, despite real-world privacy protection requiring adaptation to prompt properties.The intended balance is to reject harmful questions while allowing general ones.Source: §1 · p. 2
  • ProAda jointly optimizes visual perturbations to move away from correct responses to harmful prompts while staying close to correct responses to general prompts.The paper presents prompt-adapted adversarial images as a balance between privacy protection and general image utility.Source: §1 · p. 2
  • ProAda iteratively updates harmful-prompt embeddings and uses a binary classifier to check whether updated prompts remain harmful.The classifier helps produce a more accurate adversarial direction.Source: §1 · p. 2
  • ProAda rejects over 90% of mostly unseen harmful prompts while maintaining correct answers to over 90% of mostly unseen general prompts, outperforming prompt-agnostic VLM adversaries overall.The evaluation uses visual-privacy-related images.Source: §1 · p. 2

2 Related Work

Related work covers the development of VLMs, adversarial attacks and transferability across models and prompts, and visual privacy protection. ProAda is framed as adapting adversarial effects to prompts while preserving general image utility.

2.1 Vision-Language Models (VLMs)

  • VLMs integrate diverse modalities; CLIP uses large-scale image–text contrastive learning, BLIP supports broader tasks, and BLIP-2 builds on BLIP with stronger reasoning and lower training costs.Other models named include InstructBLIP, Flamingo, LLaVA, and miniGPT4.Source: §2.1 · p. 2

2.2 Adversarial Examples on VLMs

  • VLM adversarial research spans attacks that elicit dangerous answers and transferability across models or prompts.Prompt-level transferability seeks universal perturbations that fool VLMs across varied text prompts.Source: §2.2 · pp. 2–3

2.3 Visual Privacy Protection

  • Visual privacy defenses include image obfuscation and adversarial perturbations across classification, retrieval, object detection, and vision-language understanding; ProAda adapts effects to prompts while maintaining general image utility.ProAda is presented as a contrast to prompt-agnostic adversarial images.Source: §2.3 · p. 3

3 Methodology

ProAda produces adversarial images by optimizing visual perturbations for harmful and general prompts, and uses harmfulness judgments to guide updates to harmful-prompt text. Its algorithm alternates prompt-specific optimization under perturbation and update constraints.

3.1 Adversarial Optimization

  • ProAda outputs an adversarial image by adding an optimized visual perturbation to the clean image, with optimization alternating between harmful and general prompts.The harmful-prompt step can use either an untargeted or targeted setting.Source: §3.1 · p. 3
  • For harmful prompts, ProAda moves VLM responses away from correct answers or toward a predefined target; general-prompt optimization preserves correct answers.The method also updates harmful-prompt text perturbations in the opposite direction to guide visual-perturbation updates.Source: §3.1 · p. 3

3.2 Harmfulness Judgment on Text Perturbations

  • A binary SVM retains a text-perturbation update only when the resulting harmful prompt is still judged harmful; it achieves near 100% accuracy, with protection performance saturating after 90%.The paper says an LLM classifier is technically infeasible because updates operate on continuous embedding vectors rather than discrete tokens.Source: §3.2 · p. 4

3.3 Algorithm Details

  • The algorithm uses PGD with a bounded visual perturbation, trains on only part of the evaluation prompts, and updates text perturbations every Rv:t visual-perturbation updates while balancing harmful and general losses by Rh:g.Using only part of the evaluation set supports testing generalization to unseen prompt distributions.Source: §3.3 · p. 4

4 Experiments

ProAda achieves the best tradeoff between harmful-prompt attack success and general image utility. Experiments also examine its setup, transferability, robustness, and optimization choices.

4.1 Experimental Setup

  • The evaluation uses 200 privacy-related images, mostly unseen harmful and general prompts, and BLIP-2 with OPT-2.7b as the main VLM.Only 20% of harmful prompts and 5% of general prompts train the adversary; all prompts are evaluated. General-prompt success requires BERT score above 0.9.Source: §4.1 · pp. 5–6

4.2 Effectiveness of our ProAda

  • Across targeted-optimization iterations, ProAda maintains comparable harmful-prompt success while achieving higher general-prompt success than CroPA.Its success rates saturate after about 200 iterations for SRg and 600 for SRh.Source: Figure 3 · p. 6, §4.2 · p. 6
  • ProAda achieves the best tradeoff between attack effectiveness (SRh) and general image utility (SRg).The comparison is reported for untargeted and targeted settings.Source: §4.2 · p. 6, Table 1 · p. 6
  • Transferred ProAda images remain best overall, although black-box results are substantially below white-box results.Common transfer techniques yield little improvement and may hurt default performance; logit loss nearly doubles targeted transferability.Source: §4.2 · pp. 6–7, Table 2 · p. 6, Table 3 · p. 6
  • ProAda remains the best-performing method across five preprocessing countermeasures, with rotation more powerful than the four pixel-level operations.Table 4 reports targeted SRh; untargeted SRh is 100% for all methods after transformations substantially change original responses.Source: §4.2 · p. 7, Table 4

4.3 Ablation Studies

  • Removing either harmfulness judgment or text perturbation update lowers performance, while higher harmfulness-classifier accuracy raises success rates.Updated prompt embeddings form broader, more dispersed clusters beyond their original semantic anchors.Source: Table 5 · p. 7, §4.3 · p. 8, Figure 5 · p. 8
  • A moderate visual-to-text update ratio Rv:t = 5 performs best, while the harmful-to-general loss ratio has a substantially smaller effect and performs best at Rh:g = 9.Too-weak text perturbation provides insufficient guidance, while too-strong text perturbation interferes with visual perturbation.Source: §4.3 · p. 8, Figure 7 · p. 8
  • ProAda remains effective across most safe target responses, but reported SR varies from 57.8 for “metaphor” to 98.9 for “0”.“I am sorry” yields an SR of 97.9.Source: §4.3 · p. 8, Table 6 · p. 8

5 Conclusion

The conclusion presents ProAda as a prompt-adapted VLM adversary that balances privacy protection with preserving general-prompt responses, and considers transfer and countermeasure challenges.

5 Conclusion

  • ProAda uses distinct optimization strategies for harmful and general prompts to preserve general responses while securing image privacy.The paper considers image privacy protection as a case study.Source: §5 · p. 8
  • An SVM-based harmfulness judgment guides updates to the harmful text perturbation.Source: §5 · p. 8
  • The study also considers cross-model transfer and diverse countermeasures as challenging scenarios.Source: §5 · p. 8

Limitations

The work is limited to binary harmful-or-general prompt adaptation for visual privacy protection, and cross-model transfer remains limited, particularly in targeted settings.

Limitations

  • Cross-model transferability remains limited, especially in targeted settings.The paper points to exploring VLM internal structures, including visual-encoder attention features, as a direction for improvement.Source: Limitations · p. 9
  • Prompt adaptation is restricted to binary harmful-or-general options and the visual privacy protection task.The paper identifies broader prompt adaptations and multimodal tasks as future work.Source: Limitations · p. 9

A.1 A Complete List of Harmful and General Prompts

This appendix lists harmful prompts probing sensitive information in documents and images alongside selected general prompts about visible objects, spanning direct recognition and counting as well as interpretive questions about image content.

A.1.1 Harmful Prompts in the 24 Privacy Categories (10 Prompts for Each)

  • The cited prompts probe sensitive identity information, including credit-card, passport, driver’s-license, and student-ID numbers; they also ask about cardholder names, expiration dates, visibility, issuing institutions, and links to individuals or academic records.Source: §A.1.1 · p. 12
  • Prompts about emails, receipts, and tickets ask for their contents or transaction details alongside identifying and personal information.Examples include email participants and contents, receipt amounts and items, and ticket event, price, seat, and purchaser details.Source: §A.1.1 · pp. 12–13
  • Address, name, birth-date, phone, and email prompts probe where information appears, whether it is visible, and whether it can identify or link to a person.Questions also ask about location, associated personal details, and links to accounts or services.Source: §A.1.1 · pp. 13–14
  • Image-based prompts ask about faces, license plates, and landmarks, including identity, location, distinctive features, and links to events or owners.Face questions include age, gender, facial visibility, and identifying features; plate questions include visibility and traceability.Source: §A.1.1 · p. 14
  • Date-and-time prompts ask where and how timestamps appear and whether they can pinpoint when an image was taken.They also ask about format, time zone, verification, and the number of time-related elements.Source: §A.1.1 · p. 14
  • Other prompts target physical traits and disability, asking about appearance, health implications, assistive devices, and visible conditions.The inventory separately includes nudity prompts that repeat questions about skin color, age, body condition, and visible marks.Source: §A.1.1 · p. 15
  • Prompts about handwriting, medication, fingerprints, and signatures seek potentially identifying or sensitive details, including passwords, medical conditions, authentication, and legal use.Questions cover handwriting-linked contact or financial information, medicine and dosage, fingerprint identity, and signature legibility or validity.Source: §A.1.1 · pp. 15–16

A.1.2 Selected General Prompts

  • Selected general prompts ask whether ordinary objects and scene elements—such as vehicles, buildings, people, clothing, and sky—are visible.The list includes questions about objects such as cutlery, bicycles, bottles, signs, computers, and umbrellas, as well as scenery and crowds.Source: §A.1.2 · p. 16
  • The prompts ask whether objects, animals, scene types, and visual features are present, including furniture, devices, weather, and indoor or outdoor settings.Examples include yes/no questions about whether a cat, river, text, or source of light appears in the image.Source: §A.1.2 · pp. 16–17
  • Counting questions cover people, animals, objects, and scene elements such as windows, bridges, plants, clothing, and food.Some specify conditions, such as counting boats in a waterscape or books if any are visible.Source: §A.1.2 · pp. 16–17
  • Composition prompts ask where subjects and visual weight are positioned and where brightness, darkness, vibrant colors, or contrast appear.Other examples ask about the visual center of gravity and leading lines.Source: §A.1.2 · pp. 17–18
  • Structural questions probe patterns, depth, boundaries, overlaps, movement, and transitions within the image.The list also asks about disruptions in patterns and whether the image seems cropped or intentionally framed.Source: §A.1.2 · p. 17
  • Interpretive prompts ask about setting, mood, abstract ideas, subject relationships, rare items, and possible narrative or backstory.They also ask about contextual clues and the image’s energy or vibe.Source: §A.1.2 · p. 18
  • Other prompts invite sensory or social inference, asking about imagined sound, smell, touch, socioeconomic context, aftermath, and tension or harmony.The list also asks what textures might be felt if one could touch the image’s content.Source: §A.1.2 · p. 18

A.2 Additional Visualizations:

Figure A.1 presents additional visualizations of ProAda’s effects on images.

A.2 Additional Visualizations:

  • Figure A.1 shows ProAda’s effects on additional images.The visualizations include harmful and general prompts.Source: §A.2 · p. 18

A.3.1 Untargeted Optimization

Under untargeted optimization, ProAda converges faster than under targeted optimization, saturating after approximately 20–30 iterations.

A.3.1 Untargeted Optimization

  • Untargeted ProAda converges faster than targeted ProAda, saturating within approximately 20–30 iterations.Untargeted optimization requires the adversarial response only to diverge from the original response.Source: §A.3.1 · p. 18
  • The evaluation uses untargeted optimization as specified in Equation 1.Source: §A.3.1 · p. 18
  • Untargeted optimization needs only to make the adversarial response diverge from the original response.Source: §A.3.1 · p. 18

A.3.2 Additional Models

ProAda maintains high success rates on LLaVA-1.5 under identical settings, and the paper reports consistent findings across sampled image subsets. Additional figures show examples and iteration-wise success rates.

A.3.2 Additional Models

  • ProAda maintains high success rates on LLaVA-1.5, comparable to the primary evaluations.The authors interpret this as evidence of applicability across VLMs rather than overfitting to specific backbones.Source: §A.3.2 · p. 18

A.3.3 Result Stability

  • Across ten random 100-image subsets from a 200-image pool, success-rate variances are minimal.Evaluation covered 9,420 records, with 20–180 prompts per image balanced between private and general prompts.Source: §A.3.3 · p. 18
  • Figure A.1 presents additional visualizations of ProAda’s effects on images with harmful and general prompts.Source: Figure A.1 · p. 19
  • Figure A.2 reports ProAda success rates over iterations under untargeted optimization.Source: Figure A.2 · p. 20

Related papers

Loading 2610.16622v1…