PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Confusion Attack: Disrupting Multimodal Large Language Models

Jakub Hoscilowicz, Artur Janicki

arXiv:2511.20494v3cs.CL

TL;DR

The paper targets MLLM confusion—unstable decoding that yields hallucinations or incoherent text—by maximizing next-token entropy across an ensemble. A single perturbation disrupts the ensemble in white-box full-image and CAPTCHA settings, and the attack transfers to unseen open-source and proprietary models.

  • Problem

    Existing adversarial work targets classification errors, unsafe-content steering, or jailbreak manipulation, while confusion—unstable decoding that produces high-confidence hallucinations or incoherent text—is a distinct failure mode.

    Source: §1 · p. 1

  • Method

    The attack applies a global or localized image perturbation and maximizes next-token entropy across a small ensemble of open-source MLLMs.

    Source: §1 · p. 1, §2 · p. 2

  • Results

    A single perturbation disrupts every ensemble model in white-box full-image and Adversarial CAPTCHA settings, and transfers to unseen open-source and proprietary MLLMs.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    The authors position confusion attacks as a potential defense against unauthorized AI Agent activity, including through an Adversarial CAPTCHA or future website-UI integration.

    Source: §5 · p. 6

    The study uses entropy maximization with basic PGD; future work should examine stronger optimization methods, transformation robustness, and complex multi-step agent workflows.

    Source: §4 · p. 6

Abstract

from arXiv · show

We introduce the Adversarial Confusion Attack, a new class of threats against multimodal large language models (MLLMs). Unlike jailbreaks or targeted misclassification, the goal is to induce systematic disruption that makes the model generate incoherent or confidently incorrect outputs. Practical applications include embedding such adversarial images into websites to prevent MLLM-powered AI Agents from operating reliably. The proposed attack maximizes next-token entropy using a small ensemble of open-source MLLMs. In the white-box setting, we show that a single adversarial image can disrupt all models in the ensemble, both in the full-image and Adversarial CAPTCHA settings. Despite relying on a basic adversarial technique (PGD), the attack generates perturbations that transfer to both unseen open-source (e.g., Qwen3-VL) and proprietary (e.g., GPT-5.1) models.

1 Introduction

The paper frames confusion as a distinct MLLM failure mode: destabilizing decoding to produce high-confidence hallucinations or incoherent text. It introduces an entropy-maximizing attack and reports disruption across white-box settings and transfer to unseen models.

1 Introduction

  • Confusion targets unreliable scene understanding through unstable generation, unlike attacks aimed at classification errors, unsafe-content steering, or jailbreak manipulation.Source: §1 · p. 1
  • Prior work examines universal and patch-style attacks on aligned multimodal models, adversarial examples against proprietary systems, and scaling laws for black-box attack success.Source: §1 · p. 1
  • The Adversarial Confusion Attack maximizes output entropy to destabilize decoding and characterizes five resulting failure modes.Source: §1 · p. 1
  • A single perturbation disrupts every model in the ensemble in both full-image and Adversarial CAPTCHA white-box settings.Source: §1 · p. 1
  • The full-image attack transfers to both open-source and proprietary MLLMs.Source: §1 · p. 2

2 Method

The method perturbs an image globally or within a selected region, then optimizes next-token entropy across an ensemble of open-source MLLMs. It uses projected gradient ascent to update the perturbation.

2 Method

  • A binary mask selects the attack region; the image is perturbed within an infinity-norm bound, and pixel values are clipped to the valid range.Source: §2 · p. 2
  • The attack computes top-k probabilities from ensemble models’ final-prompt-position logits and maximizes their Shannon entropy averaged across models.Source: §2 · p. 2
  • Projected gradient ascent updates the perturbation, optionally masking the gradient to restrict updates to the patch area.Source: §2 · p. 2
  • Table 2 presents qualitative results for the white-box Adversarial CAPTCHA setup, whose original attack operates on a 1024 × 576 webpage screenshot.Source: Table 2 · p. 3

3 Experiments

The experiments measure entropy-based confusion against clean and random-noise baselines, testing full-image attacks, held-out transfer, localized patches, and proprietary models. White-box attacks increase entropy, while transfer is weaker at lower budgets.

3 Experiments

  • The setup uses four open-source models, 50 perturbation-optimization iterations, and the highest-ensemble-entropy example; CAPTCHA tests optimize a centered 224 × 224 region of a 1024 × 576 screenshot.The base screenshot is resized to 448 × 448, and the authors report no substantial result differences on other tested websites.Source: §3 · p. 3
  • Entropy is measured over the top 50 next-token logits, and ECR compares adversarial entropy with clean and random-noise baselines.Values above 1 indicate greater uncertainty than both baselines; the clean-image entropy stayed below 0.6 and was comparable to random noise.Source: §3 · pp. 3–4

3.1 Results

  • 5.08× mean ratio was the best full-image configuration; unconstrained ε=1.0 raised entropy roughly 3–6×, while ε=0.01 also increased it above baseline.The reported full-image ratios depend on learning rate, and the imperceptible perturbations have a less severe effect than unconstrained attacks.Source: §3.1 · pp. 4–5, Table 3 · p. 4
  • 1.65× mean ratio was the best unconstrained held-out transfer result, while lower budgets produced ratios near 1.1×.The cross-family protocol optimizes on two models from one family and evaluates on a held-out model from another family.Source: §3 · p. 3, Table 3 · p. 4, §3.1 · p. 5
  • 3.05× mean ratio resulted from a 224 × 224 patch that modified only approximately 9% of image pixels.Panel C evaluates a localized adversarial patch.Source: Table 3 · p. 4, §3.1 · p. 5
  • At ε=1.0, GPT-5.1, GPT-o3, GPT-4o, and Nova Pro produced coherent hallucinations, while Grok 4 issued a safety refusal.Lower-budget perturbations failed to transfer and yielded accurate descriptions of the original website.Source: §3.1 · p. 5, Table 4 · p. 5

4 Discussion

The observed confusion spans five behavioral modes, with different modes associated with different entropy levels and transfer settings. The authors note limited transfer at very small budgets and identify robustness and workflow evaluations for future work.

4 Discussion

  • Five confusion modes are reported: Blindness, Subtle, Language Switch, Delusional, and Collapse; Collapse typically coincided with peak entropy, while Subtle and Delusional modes tracked smaller increases.In proprietary black-box transfer, Collapse and Blindness were absent; Delusional hallucinations and Language Switch predominated.Source: §4 · p. 5
  • ε=0.01 perturbations were visually imperceptible but failed to transfer; simple PGD-style attacks showed limited transfer under very small budgets.The authors frame visible ε=1.0 noise patches as a reasonable denial-of-service defense for website agents, despite being conspicuous to people.Source: §4 · pp. 5–6
  • The study uses entropy maximization with basic PGD; future work should test advanced optimizers, robustness to rendering changes, and complex multi-step agent workflows.The authors also propose exploring website backgrounds or UI colors as ways to embed confusion.Source: §4 · p. 6

5 Conclusion

The conclusion presents entropy maximization with PGD and a small surrogate ensemble as a way to destabilize MLLMs globally or with a localized patch. It reports transfer in full-image tests to unseen open-source and proprietary models, and frames confusion attacks as a potential defense against unauthorized AI-agent activity.

5 Conclusion

  • The Adversarial Confusion Attack disrupts MLLMs by maximizing next-token entropy.Source: §5 · p. 6
  • Using standard PGD and a small surrogate ensemble, one perturbation can destabilize decoding when applied globally or as a localized patch.Source: §5 · p. 6
  • In full-image tests, the attack transfers to unseen open-source and proprietary models; the authors position it as a potential defense against unauthorized AI-agent activity.Source: §5 · p. 6

A Appendix: Qualitative Results

The appendix presents proprietary-model screenshots showing behaviors ranging from refusals and noise detection to hallucinations and fabricated scene descriptions.

Loading 2511.20494v3…