Source-linked AI summary
Sparse Autoencoders as Plug-and-Play Firewalls for Adversarial Attack Detection in VLMs
Hao Wang, Yiqun Sun, Pengfei Wei, Lawrence B. Hsieh, Daisuke Kawahara
TL;DR
VLMs are increasingly deployed but remain vulnerable to adversarial attacks, while existing safety attention and defenses are limited. The paper introduces SAEgis, a plug-and-play sparse-autoencoder detector that identifies attack-relevant latent features without additional adversarial training. SAEgis performs strongly across in-domain, cross-domain, and cross-attack settings, with multi-layer signal fusion further improving robustness and stability.
Problem
VLM safety has received limited attention despite the models’ vulnerability to adversarial attacks in real-world applications.
Method
SAEgis inserts an SAE into a pretrained VLM, trains it with reconstruction, selects top-k attack-relevant features, and detects attacks from their activation patterns.
Results
SAEgis performs strongly across in-domain, cross-domain, and cross-attack settings, while multi-layer ensembling further improves performance and stability.
Takeaways & Limitations
Sparse latent features provide a practical foundation for adversarial detection and improving the safety of real-world VLM systems.
Takeaways & Limitations
Under cross-domain or cross-attack shifts, distribution changes can miscalibrate thresholds and degrade detection performance.
Abstract
from arXiv · showhide
Vision-language models (VLMs) have advanced rapidly and are increasingly deployed in real-world applications, especially with the rise of agent-based systems. However, their safety has received relatively limited attention. Even the latest proprietary and open-weight VLMs remain highly vulnerable to adversarial attacks, leaving downstream applications exposed to significant risks. In this work, we propose a novel and lightweight adversarial attack detection framework based on sparse autoencoders (SAEs), termed SAEgis. By inserting an SAE module into a pretrained VLM and training it with standard reconstruction objectives, we find that the learned sparse latent features naturally capture attack-relevant signals. These features enable reliable classification of whether an input image has been adversarially perturbed, even for previously unseen samples. Extensive experiments show that SAEgis achieves strong performance across in-domain, cross-domain, and cross-attack settings, with particularly large improvements in cross-domain generalization compared to existing baselines. In addition, combining signals from multiple layers further improves robustness and stability. To the best of our knowledge, this is the first work to explore SAE as a plug-and-play mechanism for adversarial attack detection in VLMs. Our method requires no additional adversarial training, introduces minimal overhead, and provides a practical approach for improving the safety of real-world VLM systems.
1 Introduction
VLMs are increasingly important in real-world applications, yet their safety has received limited attention and they remain vulnerable to adversarial image attacks. SAEgis addresses this gap with a plug-and-play SAE-based detector that identifies attack-relevant latent features and performs strongly across evaluation settings.
- VLMs have expanded from image-description tasks to visual reasoning and embodied AI, increasing their role in real-world applications.
- Inputs activating many attack-relevant features are flagged as adversarial, while inputs activating few are classified as clean.
- Adversarial perturbations can compromise VLMs, motivating attack methods targeting increasingly capable systems.
- SAEgis inserts an SAE into a pretrained VLM, trains it with reconstruction, and uses top-k attack-relevant features to detect adversarial inputs.
- The framework requires no additional adversarial training, is plug-and-play, and adds minimal computational overhead to the original VLM.
- SAEgis performs strongly in in-domain, cross-domain, and cross-attack settings, with particularly better cross-domain generalization than existing baselines.
2 Related Work
Related work has established that VLMs are vulnerable to transferable adversarial attacks and has developed increasingly efficient attack-generation methods. Defensive research has explored reconstruction, probing with attention maps, and learned steering vectors for attack detection or mitigation.
- Transfer-based attacks use adversarial images generated on models such as CLIP and BLIP to attack other VLMs.
- SSA-CWA improves attack transferability by combining Spectrum Simulation Attack with Common Weakness Attack.
- AdvDiffVLM uses diffusion models to generate adversarial examples more efficiently.
- MirrorCheck detects attacks by comparing embeddings of original images with images reconstructed from generated captions.
- PIP trains an SVM using attention maps derived from irrelevant probe questions to classify adversarial inputs.
- Other defensive work constructs adversarial datasets and learns steering vectors for VLM attack defense.
3 Methodology
SAEgis detects adversarial inputs by selecting SAE latent features whose activations differ between clean and attacked images, then thresholding their activation counts using clean data. A multi-layer ensemble averages layer-specific statistics and preserves clean-only calibration while improving stability across evaluation settings.
- Framework overview: SAEgis inserts an SAE into a pretrained VLM and uses sparse latent activations to identify attack-relevant signals.The SAE may be placed in the vision encoder, projection layer, or language model.
- Attack-Relevant Feature Selection: The feature score combines activation peak strength with spatial extent across image tokens, balancing localized and broadly distributed perturbations.A logarithm prevents broadly distributed activations from dominating while retaining signal from strong localized attacks.
- Attack-Relevant Feature Selection: Attack relevance is computed as the difference between mean feature scores on adversarial and clean images, and the top-K features are selected without training a classifier.This difference-of-means approach avoids additional optimization and scaling costs associated with classifiers over all SAE features.
- Adversarial Detection: Detection thresholds are calibrated solely on a held-out clean development set by taking a target false-positive-rate quantile of attack-relevant feature activation counts.At inference, inputs exceeding the threshold are classified as adversarial; for example, α can be set to 0.02.
- Multi-Layer SAE Ensembling: The multi-layer ensemble uniformly averages layer-specific activation statistics and retains clean-only threshold calibration, improving performance and stability across in-domain, cross-domain, and cross-attack settings.The ensemble uses SAE modules at a set of layers, with each layer retaining its own attack-relevant feature set.
4 Experiments
The experiments evaluate SAEgis across in-domain, cross-domain, and cross-attack settings using standardized image-only detection metrics and multiple attack methods. Results show strong overall performance, especially under domain shift, while layer choice and ensembling materially affect robustness.
- Evaluation Protocol: SAEgis is evaluated as an image-only binary detector using precision, recall, and F1-score at a controlled false-positive target of α = 0.02.The threshold is calibrated on clean development data before testing.
- Evaluation Protocol: Experiments cover in-domain, cross-domain, and cross-attack settings across NIPS17, LLaVA, and Medical datasets.The attacks include SSA-CWA, M-Attack, and FOA-Attack, with cross-attack transfer from SSA-CWA to stronger attacks.
- Implementation Choices: Vision-block0, vision-block10, and projection-mlp2 are the strongest SAE insertion locations among nine candidates, while at least 128 attack-relevant features are suggested.Early vision layers capture high-frequency patterns, whereas projection-mlp2 interfaces with the language model and may encode more global information.
- Main Results: In-domain results are strong for all methods, with Dense (Ensemble) and SAEgis (Ensemble) performing best overall.Multi-layer ensembling substantially improves recall, although SAEgis is slightly worse than the dense baseline on Medical.
- Main Results: Cross-domain transfer substantially challenges baselines, whereas ensembled SAEgis keeps precision and recall above 90% in most cases.The challenge occurs in both transfers from common to specialized domains and transfers from specialized to general domains.
- Main Results: Cross-attack transfer degrades projection-mlp2-based detection, but vision-block0 remains strong and multi-layer ensembling recovers performance comparable to in-domain results.The authors hypothesize that early layers capture attack-consistent high-frequency textures and edges, while projection-layer signals vary more across attacks.
- Main Results: Overall, SAEgis maintains strong performance across all three evaluation settings, indicating that SAE features capture attack-relevant signals beyond the training domain and attack strategy.This supports its use as a foundation for adversarial detection under distribution shift.
5 Analysis
SAEgis exhibits transferable attack-relevant features across datasets and attack methods, while layer choice affects robustness under distribution shifts. Failure cases and ablations motivate combining complementary layer signals.
- Feature Transferability: Substantial feature overlap across datasets suggests SAEgis captures transferable attack-relevant representations, including across the distinct Medical domain.The overlap analysis links this transferability to the observed cross-domain performance.
- Attack-Method Overlap: At vision-block0, M-Attack and FOA-Attack show nearly complete feature overlap, while SSA-CWA overlap with the other attacks decreases at greater layer depth.The pattern is consistent with local perturbation features and more global feature shifts being represented at different depths.
- Failure Cases: Cross-domain and cross-attack distribution shifts can move clean or adversarial activation counts across the calibration threshold, reducing precision or recall.Clean shifts to higher counts can cause false positives, whereas shifts to lower counts can cause adversarial images to be classified as clean.
- Multi-Layer Ensembling: Ensembling signals from vision-block0, deeper layers, and projection-mlp2 yields additional gains, with the three-layer ensemble performing best overall.The ablation attributes the improvement to complementary representations at different abstraction levels.
6 Conclusion
The paper introduces SAEgis, a plug-and-play sparse-autoencoder framework for detecting adversarial inputs in VLMs without adversarial training. It reports strong performance across evaluation settings and improved stability from fusing signals across layers.
- Contribution: SAEgis uses sparse autoencoders as plug-and-play modules for adversarial attack detection in vision-language models.The framework leverages sparse latent features to identify attack-relevant signals.
- Results: SAEgis achieves strong performance across in-domain, cross-domain, and cross-attack settings without requiring adversarial training.The conclusion presents this as the framework's central empirical result.
- Implication: Fusing signals across layers improves SAEgis stability and performance, with each layer contributing complementary representations at different abstraction levels.The conclusion characterizes sparse latent features as a practical foundation for enhancing VLM safety.