PaperlayerView exact revision on arXiv

Source-linked AI summary

Less Is More -- Until It Breaks: Security Pitfalls of Vision Token Compression in Large Vision-Language Models

Xiaomei Zhang, Zhaoxi Zhang, Leo Yu Zhang, Yanjun Zhang, Guanhong Tao, Shirui Pan

arXiv:2601.12042v1cs.CRcs.AI

TL;DR

Visual token compression improves LVLM efficiency, but its security implications are underexplored because compression can expose robustness failures absent from uncompressed inference. This paper analyzes the mechanism, proposes compression-aware attacks including a black-box transfer variant, and finds severe compression-specific degradation with limited defense protection. The results reveal an efficiency-security trade-off in efficient LVLMs.

  • Problem

    The robustness effects of visual token compression remain largely unexplored, and existing attacks do not isolate failures arising exclusively under compressed inference.

  • Method

    The paper analyzes token-importance ranking instability and develops Compression-Aware Attack, extending it to Transfer CAA when the target model and compression configuration are unknown.

  • Results

    Visual token compression introduces severe, compression-specific robustness degradation across models, benchmarks, and compression methods, while existing defenses provide limited protection.

  • Takeaways & Limitations

    Efficient LVLM deployment requires security-aware compression designs that balance inference efficiency with robustness.

  • Takeaways & Limitations

    The black-box transfer setting assumes no access to target parameters, internal states, or gradients and restricts interaction to a query-limited API.

Abstract

from arXiv · show

Visual token compression is widely adopted to improve the inference efficiency of Large Vision-Language Models (LVLMs), enabling their deployment in latency-sensitive and resource-constrained scenarios. However, existing work has mainly focused on efficiency and performance, while the security implications of visual token compression remain largely unexplored. In this work, we first reveal that visual token compression substantially degrades the robustness of LVLMs: models that are robust under uncompressed inference become highly vulnerable once compression is enabled. These vulnerabilities are state-specific; failure modes emerge only in the compressed setting and completely disappear when compression is disabled, making them particularly hidden and difficult to diagnose. By analyzing the key stages of the compression process, we identify instability in token importance ranking as the primary cause of this robustness degradation. Small and imperceptible perturbations can significantly alter token rankings, leading the compression mechanism to mistakenly discard task-critical information and ultimately causing model failure. Motivated by this observation, we propose a Compression-Aware Attack to systematically study and exploit this vulnerability. CAA directly targets the token selection mechanism and induces failures exclusively under compressed inference. We further extend this approach to more realistic black-box settings and introduce Transfer CAA, where neither the target model nor the compression configuration is accessible. We further evaluate potential defenses and find that they provide only limited protection. Extensive experiments across models, datasets, and compression methods show that visual token compression significantly undermines robustness, revealing a previously overlooked efficiency-security trade-off.

1 Introduction

Visual token compression improves LVLM inference efficiency but introduces robustness vulnerabilities that are absent under full-token inference. The paper identifies unstable token-importance rankings as the mechanism and proposes attacks that exploit this compression-specific weakness.

  • Motivation: Visual token compression is motivated by visual redundancy and the task-specific nature of relevant evidence, enabling retention of only highly ranked tokens.Text-guided methods use the prompt to reduce tokens while aiming to preserve task-relevant information.
  • Security gap: Compression changes LVLM robustness behavior: models robust to small perturbations with full tokens become vulnerable when compression is activated, especially at lower retention rates.This creates risks when dynamic systems automatically adjust retention according to load or computational budgets.
  • Security gap: Existing adversarial attacks target fixed inference settings and do not isolate failures caused exclusively by compression, while token selection is non-differentiable.These properties limit the applicability of classical end-to-end gradient-based attacks.
  • Approach: Compression-Aware Attack manipulates importance rankings so compression retains less relevant tokens and discards task-critical evidence while minimally affecting uncompressed inference.Its components include selective perturbation of low-importance regions, hierarchical ranking optimization, and semantic erasure of surviving tokens.
  • Approach: Transfer CAA extends the attack to black-box settings by generating surrogate-model adversarial examples that manipulate token importance across plausible compression configurations.It uses an uninformative image border and universal templates to force retention of uninformative tokens.
  • Findings: 47.61% average Compression Sensitivity Gap was induced by CAA, compared with 2.36% for baseline attacks across models, benchmarks, and compression methods.The results expose an efficiency-security trade-off and potential defenses provide limited protection.

2 Background

LVLMs combine visual encoders and language models to process multimodal inputs, while vision token compression reduces inference cost by ranking and retaining selected visual tokens. The compression mechanism is defined by its placement, importance scoring, and retention schedule.

  • LVLM architecture: LVLMs encode image patches as visual tokens, concatenate them with text tokens, and process the combined sequence through transformer layers to generate text.The vision encoder and projection module produce vision tokens whose hidden dimension matches the language model representation.
  • Efficiency motivation: Self-attention dominates transformer-based LVLM computation, motivating compression to reduce the cost of processing many visual tokens.Compression is presented as an inference-time approach that does not require retraining.
  • Compression approaches: Text-guided compression uses textual context to reduce visual tokens more aggressively while targeting task-relevant evidence, unlike text-agnostic pruning or merging.The paper focuses on attention-based importance estimation for text-guided compression.
  • Compression formulation: A compression mechanism specifies target layers, an importance function, and a retention schedule defining the fraction of original visual tokens retained at each layer.For input (𝐼,𝑇), the mechanism C = {˜𝐿, R, 𝐹s} produces compressed inference output ˜Y = f(𝐼,𝑇; C).
  • Compression operation: At each compression layer, the operator scores visual tokens and retains the top ⌈r(˜l)nV⌉ tokens, while text tokens remain unchanged.The selected visual tokens are restored to their original order before forming the next-layer input.
  • Compression operation: Compressed visual sequences provide the input to subsequent layers, and repeated compression operates on tokens already compressed by earlier layers.The retention rate controls the visual token count at each designated compression layer.

3 Demystifying Robustness Degradation Induced by Compression

Visual token compression degrades LVLM robustness because perturbations destabilize token-importance rankings, causing critical tokens to be discarded. Ranking instability is reflected in deteriorating global and local ranking metrics and can produce severe prediction collapse.

  • Robustness degradation: Compression degrades robustness, and token ranking is identified as the primary source of this degradation.The analysis examines compression’s pipeline after empirical evidence establishes the robustness problem.
  • Ranking-driven failures: Restoring the clean token ranking under perturbed inputs substantially recovers performance toward the clean baseline.The comparison separates ranking errors from semantic corruption in retained tokens.
  • Selection consequences: Retaining Bottom-k tokens instead of Top-k tokens causes a complete performance collapse under compression.The extreme reversed-selection experiment demonstrates the model’s dependence on correct token selection.

4 The Proposed Compression-Aware Attack

The proposed attacks exploit compression-specific failures by manipulating token-importance rankings while preserving behavior without compression. CAA targets known compression stages, whereas T-CAA transfers this strategy without access to the target model or compression configuration.

  • CAA aims to preserve non-compressed behavior while inducing failures only when visual token compression is enabled.The attack objective explicitly separates compressed and uncompressed outputs.
  • CAA targets the first compression stage because ranking errors there are irreversible and propagate through subsequent stages.The attack disrupts importance ranking so compression retains minimally informative tokens.
  • Selective perturbation modifies only low-importance regions, avoiding high-importance tokens that encode core semantic evidence for correct predictions.This preserves full-sequence behavior while redirecting compressed selection toward perturbed regions.
  • The hierarchical ranking objective promotes low-importance tokens above important ones and reverses their internal ordering, preventing compression from retaining the most informative candidates.It combines inter-group manipulation with intra-group reversal.
  • Semantic erasure corrupts information in tokens selected after ranking manipulation, complementing the attack by making retained tokens semantically uninformative.CAA also uses query-guided key alignment to stabilize and accelerate optimization.
  • T-CAA addresses unknown target models and compression configurations using transfer attacks that optimize perturbations across plausible compression layers.Model mismatch makes least-important regions unreliable, motivating a border-based strategy.

5 Experiments

Experiments across LVLMs, benchmarks, compression methods, and attack settings show that CAA selectively degrades compressed inference while preserving uncompressed performance. Analyses and ablations attribute this behavior to manipulated token rankings and the coordinated design of CAA’s components.

  • Experimental Setup: CAA evaluated LLaVA, LLaVA-Next, and Qwen2.5-VL on POPE, MME, and TextVQA using FastV, PDrop, and SparseVLM.The evaluation covers three victim models, three benchmarks, and three representative compression methods.
  • Evaluation Metrics: UPR, CAE, and CSG measure retained uncompressed performance, compressed attack degradation, and the compression-induced performance gap.These metrics compare clean and adversarial inputs under compressed and non-compressed inference.
  • Attack Performance: 56.60% average compressed-performance drop, 91.99% UPR, and 47.61% average CSG show that CAA causes compression-specific failures at retention rate 0.2.Vanilla attacks mainly disrupt uncompressed inference, while random attacks achieve only 2.36% average CSG.
  • Attack Performance: CAA promotes irrelevant Bottom-100 tokens into Top-100 selections; compression makes this ranking distortion irreversible, discarding essential visual evidence.Under uncompressed inference, later layers can reallocate attention, but compressed inference permanently removes discarded tokens.
  • Ablation Studies: CAA-Full reduces average UPR from 0.9152 to 0.7691 and yields 0.3475 CSG, whereas Most-Only reaches CAE up to 0.7427 but only 0.1919 average CSG.The results support selectively perturbing low-importance regions rather than perturbing the whole image or only high-importance regions.
  • Ablation Studies: Removing BPR lowers average CAE from 0.6691 to 0.1715 and reduces CSG by 0.4482, while removing hierarchical ranking lowers CAE to 0.5714 and CSG to 0.4954.These ablations show that relative ranking optimization and hierarchical ranking constraints materially contribute to attack effectiveness.
  • Black-box Evaluation: T-CAA achieves 29.97% average CSG across transfer pairs without target-model or compression-configuration access, and outperforms HSJA and RayS under a 600-query budget.Transfer attacks use surrogate models and target consistent token-importance manipulation across plausible compression layers.

6 Discussion

CAA perturbs least-informative regions to alter token-importance rankings without changing overall image semantics, causing failures only after compression. This stealthiness defeats both visual-signal and prediction-shift detection approaches.

  • Detection: CAA’s semantic preservation makes clean and adversarial samples overlap in CLIP-similarity distributions after image transformations.The evaluated transformations include JPEG compression and Gaussian blur.
  • Detection: Prediction-shift detection also fails because CAA preserves uncompressed behavior while triggering failures only under compression.The attack’s state-specific effect limits detection based on end-to-end LVLM outputs.

7 Conclusion

The paper systematically studies robustness risks created by visual token compression and introduces attacks that expose failures specific to compressed inference. Its findings identify an efficiency–security trade-off and motivate security-aware compression design.

  • Conclusion: CAA exploits fragile token-importance rankings to induce compression-specific failures while largely preserving uncompressed behavior.T-CAA extends this approach to black-box settings without requiring target-model or compression-configuration knowledge.
  • Conclusion: The results show that compression can introduce stealthy, hard-to-diagnose failure modes and that existing defenses provide limited protection.This exposes an overlooked efficiency–security trade-off in efficient LVLMs.
  • Implications: The paper recommends considering security and robustness alongside efficiency when designing and deploying compressed LVLM systems.It identifies security-aware compression mechanisms and robustness-aware mitigation strategies as directions for future work.
  • Conclusion: Visual token compression retains selected visual tokens to reduce computational cost, but its security implications and robustness effects have been insufficiently examined.Compression methods exploit spatial redundancy or prompt-relevant visual evidence to discard tokens.

D.1 The Impact of Token Ranking on Robustness

Ranking instability under perturbation causes irrelevant tokens to enter the retained set, while restoring the clean ranking substantially recovers performance. This supports ranking instability as a mechanism of compression-related robustness degradation.

  • The Impact of Token Ranking on Robustness: Restoring clean token-importance rankings produces substantial performance recovery compared with using perturbed rankings, remaining close to the clean baseline.The comparison is reported under the same three inference configurations and a perturbation budget of 64/255.
  • The Impact of Token Ranking on Robustness: Increasing random-noise magnitude progressively disorders Qwen-VL’s rankings, bringing originally Bottom-100 tokens into the Top-100 set.The resulting misranking retains uninformative tokens while excluding task-critical ones.

E Victim Models and Datasets

The evaluation covers three LVLMs and multimodal benchmarks, with filtering and attack procedures designed to test compression-specific failures requiring genuine visual reasoning.

  • The evaluation uses LLaVA, LLaVA-Next, and Qwen2.5-VL across POPE, MME, and related multimodal benchmarks.POPE measures object hallucination through binary visual question answering, while MME assesses multimodal perception and reasoning.
  • Samples are retained when the model prediction changes between image-present and image-absent inputs, ensuring genuine multimodal reasoning is required.
  • CAA injects perturbations into unimportant image regions to manipulate compression into retaining uninformative tokens, causing failure under compressed inference.
  • The vanilla attack loss suppresses predefined correct-answer variants and promotes incorrect anchor answers using the first generated token’s output distribution.
  • The perturbation constraint is ∥δ∥∞≤ε, with ε set to 32/255 by default.

F.2 Main Results for CAA.

CAA preserves performance without compression while substantially degrading performance under compression, demonstrating that it exposes vulnerabilities introduced by the compression mechanism across compression strengths.

  • CAA consistently achieves high CSG values under PDrop and across tested retention rates on TextVQA.The reported evaluations indicate effectiveness across varying retention rates.
  • Algorithm 1 defines CAA as an attack procedure producing an adversarial image from attack hyperparameters and hierarchy depth.
  • CAA computes clean token importance, separates most- and least-important indices, and partitions least-important indices into hierarchical groups.
  • CAA preserves performance in the non-compressed state while inducing substantial degradation under compression.

F.3 Ablation Study of Key Components in CAA

Ablations and sensitivity analyses show that CAA depends on its ranking, erasure, and alignment components, while effectiveness varies with configuration mismatch and perturbation budget.

  • F.3 Ablation Study of Key Components in CAA: Removing hierarchical ranking, semantic erasure, or query-guided alignment consistently reduces CAA effectiveness under compressed inference on Qwen-VL.Removing hierarchical ranking or query-guided optimization causes substantial CSG drops, while removing semantic erasure leaves retained tokens usable.
  • F.3 Ablation Study of Key Components in CAA: CAA remains highly effective when attack and test retention rates differ modestly and is robust to a certain degree of layer mismatch.
  • F.3 Ablation Study of Key Components in CAA: As ε increases across 16/255, 32/255, and 64/255, attack effectiveness improves consistently across models and datasets.The perturbations remain largely imperceptible because they are restricted to a small image portion.
  • F.3 Ablation Study of Key Components in CAA: At larger perturbation budgets, visible artifacts and degraded uncompressed inference reduce overall CSG.This creates a trade-off between attack strength and perceptual stealth.
  • F.3 Ablation Study of Key Components in CAA: Different models substantially disagree on least-important regions, making direct transfer of surrogate white-box attacks suboptimal.
  • F.3 Ablation Study of Key Components in CAA: Table 13 reports T-CAA across models and compression configurations, while Table 14 ablates least-important-region perturbation, full-border optimization, and suppression templates.
  • F.3 Ablation Study of Key Components in CAA: For average retention rate r̄≤0.2, the selected attack layer range is layers 2–3, where retention first enters the attack-effective regime.Candidate compression configurations are enumerated under deployment-consistent rate and interval constraints.

G.3 Main Results for Transfer Attack

T-CAA transfers compression-aware attacks across models and unknown compression configurations by manipulating token-importance ordering while preserving non-compressed behavior.

  • T-CAA jointly learns border-amplification and content-suppression templates to steer compression toward uninformative tokens.
  • UPR averages 0.8550 while CAE averages 0.4445 across models and compression configurations on POPE.T-CAA preserves model behavior without compression and causes severe degradation once compression is enabled.

G.4 Ablation Study of Key Components in T-CAA

The ablation study examines attack-region selection, universal perturbation templates, and the down template in T-CAA's black-box transfer setting. Results show that surrogate-selected regions, direct full-border optimization, and removing the down template each weaken transferability or attack effectiveness.

  • Ablation Setup: The ablation evaluates attack-region selection, universal perturbation templates versus full-border optimization, and the down template in the black-box setting.These three components are tested to determine their contribution to T-CAA transferability and effectiveness.
  • Attack Region Selection: Surrogate-selected least regions can preserve target-relevant information, degrading non-compressed inference and weakening attack effectiveness under compression.Model-specific importance estimation means regions judged unimportant by a surrogate may remain important to the target model.
  • Full-Border Optimization: Directly optimizing the entire border produces substantially lower average CSG than T-CAA, indicating weaker transferability.The resulting perturbations are surrogate-specific, whereas universal templates capture more model-agnostic patterns and transfer more reliably.
  • Down Template: Removing the down template reduces attack effectiveness on compressed models, yielding average CAE = 0.1336.Boosting border-token importance alone is insufficient for reliable transfer because cross-model attention differences can prevent target models from reproducing the intended token-ranking shift.
Loading 2601.12042v1…