PaperlayerView exact revision on arXiv

Source-linked AI summary

Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models

Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge Belongie, Zeynep Akata

arXiv:2504.02821v3cs.CVcs.AIcs.LG

TL;DR

VLM representations remain difficult to interpret at the neuron level because concepts can be entangled and neurons can be polysemantic. The paper introduces MS and human-aligned evaluation for VLM SAEs, then uses SAE interventions to steer multimodal outputs. It finds improved monosemanticity, especially with sparsity and wider latents, while noting that high MS does not always yield precise steering effects.

  • Problem

    VLM neurons can encode multiple unrelated concepts, while practical neuron-level metrics for evaluating SAE disentanglement remain limited.

  • Method

    The paper trains SAEs on VLM representations, evaluates neuron monosemanticity with activation-weighted image similarity and a human-study benchmark, and intervenes on CLIP SAE neurons for MLLM steering.

  • Results

    SAEs significantly increase neuron monosemanticity; wider and sparser latents improve MS, and Matryoshka SAEs achieve overall superior scores.

  • Takeaways & Limitations

    SAEs provide an unsupervised route to interpreting VLM features and steering multimodal outputs through the vision encoder.

  • Takeaways & Limitations

    High-MS neurons do not always produce precise MLLM steering effects, partly because MLLMs may lack fine-grained understanding or alignment with the vision encoder.

Abstract

from arXiv · show

Sparse Autoencoders (SAEs) have recently gained attention as a means to improve the interpretability and steerability of Large Language Models (LLMs), both of which are essential for AI safety. In this work, we extend the application of SAEs to Vision-Language Models (VLMs), such as CLIP, and introduce a comprehensive framework for evaluating monosemanticity at the neuron-level in visual representations. To ensure that our evaluation aligns with human perception, we propose a benchmark derived from a large-scale user study. Our experimental results reveal that SAEs trained on VLMs significantly enhance the monosemanticity of individual neurons, with sparsity and wide latents being the most influential factors. Further, we demonstrate that applying SAE interventions on CLIP's vision encoder directly steers multimodal LLM outputs (e.g., LLaVA), without any modifications to the underlying language model. These findings emphasize the practicality and efficacy of SAEs as an unsupervised tool for enhancing both interpretability and control of VLMs. Code and benchmark data are available at https://github.com/ExplainableML/sae-for-vlm.

1 Introduction

VLM internals remain difficult to interpret because neurons can represent multiple unrelated concepts. This work evaluates SAEs as a way to disentangle visual concepts, measure neuron-level monosemanticity, and steer multimodal outputs.

  • Motivation: VLMs such as CLIP have broad applications, but their internal representations remain insufficiently understood for systematic analysis.The paper motivates methods that can probe representations in models used across visual and textual modalities.
  • Motivation: Polysemantic neurons can respond strongly to unrelated concepts because concepts may be encoded as combinations of neurons.SAEs reconstruct activations in a higher-dimensional space to separate overlapping concepts.
  • Contributions: The MonoSemanticity score measures activation-weighted pairwise similarity among images that strongly activate a neuron.Higher similarity indicates that the neuron is more narrowly focused on one concept.
  • Contributions: SAE neurons are more monosemantic than original VLM neurons, with examples shifting from mixed objects such as cellphones and rulers toward focused concepts such as parrots.Figure 1 contrasts polysemantic pretrained-VLM neurons with monosemantic SAE latents using MS.
  • Contributions: Intervening on one monosemantic SAE neuron in a CLIP vision encoder can insert or suppress its concept in an MLLM output without changing the language model.The paper demonstrates this transfer using CLIP-based SAEs and models such as LLaVA.
  • Contributions: The study validates MS against human perception, compares SAE architectures and latent designs, and reports stronger scores for wider and sparser latents.The contributions identify Matryoshka SAE as having overall superior MS.

2 Related Work

Prior work established SAEs for interpreting language, vision, diffusion, and multimodal models, while this paper focuses on neuron-level monosemanticity in VLMs. Related research also includes improved SAE mechanisms and shared concept analysis across vision encoders.

  • Sparse Autoencoders: Recent work repurposes dictionary learning and SAEs to improve interpretability in both language and vision-language models.The related literature also studies SAE-based steering of language models.
  • Sparse Autoencoders: SAE mechanisms have been enhanced with activation functions such as Batch TopK and JumpReLU, alongside Matryoshka representation learning.These developments form part of the methodological context for the paper’s SAE analysis.
  • Vision-Language Models: Vision-language models align images and text in shared embedding spaces or generate text conditioned on images, with CLIP among the foundational examples.The literature spans models with aligned representations and models that produce multimodal outputs.
  • Vision-Language Models: Prior studies examine internal representations of VLMs because trust, understanding decision-making, and steering have become important concerns.This motivates interpretability research beyond benchmark performance.
  • SAEs for VLMs: SAEs have been applied to CLIP, DINOv2, diffusion models, and multimodal LLMs, while other work studies common concepts across vision encoders.The paper positions its VLM monosemanticity analysis within this expanding application area.

3 Sparse Autoencoders for VLMs

Sparse autoencoders learn sparse, overcomplete representations of VLM embeddings, enabling neuron-level measurement of monosemanticity and targeted interventions in multimodal models.

  • SAE formulation: SAEs decompose model embeddings into sparse activations over an overcomplete latent dictionary and reconstruct the original embedding.The latent width is set as an expansion factor times the original dimension.
  • SAE formulation: The SAE loss combines reconstruction quality with a sparsity penalty weighted by λ, while TopK and Matryoshka variants impose alternative sparsity or reconstruction structures.Matryoshka SAEs train nested dictionaries using multiple reconstruction objectives and can be combined with other SAE variants.
  • Monosemanticity Score: MonoSemanticity measures whether images strongly activating a neuron are semantically similar, using activation-weighted pairwise image similarities.The score is computed neuron-wise after extracting image embeddings and normalizing neuron activations.
  • Monosemanticity Score: Higher MS scores correspond to more similar top-activating images, indicating that the neuron responds more consistently to a single concept.Examples range from same-object images at MS = 0.9 to less similar or different objects as scores decrease.
  • Steering MLLMs with Vision SAEs: A CLIP-based SAE can intervene on a selected neuron after the vision encoder, decode modified token embeddings, and steer LLaVA outputs without changing the language model.The intervention is applied across visual token embeddings before they are processed normally by the multimodal model.

4 Experiments

The experiments evaluate SAEs across several vision encoders and test whether their learned features improve monosemanticity. Results compare SAE variants, expansion factors, human alignment, and original neurons.

  • Setup: SAEs are trained on activation vectors from CLIP, SigLIP, AIMv2, and WebSSL vision encoders, using final-layer or specified token embeddings.Steering experiments use activation vectors from two random token embeddings at layer l = 22 when no CLS token is used.
  • Setup: The study compares BatchTopK and Matryoshka BatchTopK SAEs across expansion factors ε ∈ {1, 2, 4, 8, 16, 64}, with K = 20 by default.Matryoshka groups are set to M = {0.0625ω, 0.1875ω, 0.4375ω, ω}.
  • Human validation: Human alignment with MS is evaluated by asking which neuron in a pair is more monosemantic, grouped by their MS difference.Bars report alignment rate within each difference interval, while dots report cumulative alignment up to that interval.
  • Monosemanticity results: SAE neurons consistently achieve higher best-neuron MonoSemanticity scores than original neurons, and higher expansion factors generally increase the highest MS.At ε = 1, improvements already indicate that sparse dictionary learning contributes beyond increased dimensionality; Matryoshka reconstruction improves separation but lowers R2 by about 2 or 3 points at matched expansion factors.
  • Monosemanticity results: Matryoshka SAE improvements in top MS scores are consistent across vision encoders trained with different objectives.This comparison supports evaluating SAE features across multiple vision representations rather than only one encoder.

No SAE

Across the reported comparisons, SAE features improve monosemanticity and support concept steering in LLaVA. The experiments also show trade-offs between sparsity, reconstruction quality, and steering behavior.

  • Monosemanticity: SAE neurons improve MS across most neurons, with about 90% outperforming original neurons even at equal width ε = 1.Across-neuron MS rises overall with expansion factor through ε = 4, then decreases for wider latents, although their absolute number of high-scoring neurons still increases.
  • Monosemanticity: Stricter sparsity constraints produce more monosemantic features overall, but K = 1 is not always optimal because higher MS can reduce reconstruction quality.R2 ranges from 31.3% at K = 1 to 74.9% at K = 50; K = 20 gives R2 = 66.8% in the same setup.
  • Steering evaluation: Table 2 compares the percentage of generations satisfying concept-insertion and suppression criteria for SAE steering directions and DiffMean.The table reports generation-level evaluation outcomes for both steering directions.
  • Concept insertion: Increasing activation of CLIP SAE neuron #39, associated with pencils, shifts LLaVA’s output from following a poem instruction toward describing pencil attributes and the pencil itself.The intervention changes the multimodal output while the underlying language model remains unmodified.
  • Concept suppression: Decreasing activations of knife- and laptop-associated neurons progressively suppresses those concepts while LLaVA continues describing other visible objects.The intervention first produces substitutions such as wooden utensils or a computer monitor, then omits the targeted concepts.

5 Conclusion

The paper introduces MS for evaluating neuron-level monosemanticity in VLM SAEs, validates it against human perception, and demonstrates SAE-based concept steering in multimodal LLMs. It reports that sparsity and wider latents improve monosemanticity, while high MS does not always yield precise steering effects.

  • The MonoSemanticity score evaluates neuron-level monosemanticity in SAEs trained on VLMs.The metric incorporates human judgment into evaluating dictionary-learning methods.
  • Sparsity and wider latents primarily increased monosemanticity, with Matryoshka SAEs showing superior performance.
  • SAE concepts enabled unsupervised, concept-based steering of multimodal LLM outputs.
  • The authors propose adapting MS to text representations and studying interactions between specialized and broad learned concepts.
  • High MS neurons did not always produce precise MLLM steering effects, partly because MLLMs may lack fine-grained understanding or align imperfectly with vision encoders.Some feature-detector neurons also showed no clear steering effect.

B More details on steering

The appendix illustrates how SAE neurons steer LLaVA-like outputs by modifying concept-related activations after the vision encoder. Examples vary intervention strength and use diverse prompts to show concept insertion or suppression.

  • An SAE attached after the vision encoder lets a single concept neuron steer the LLM response toward a concept absent from the input image.The figure uses panda as an example and attributes this flexibility to the SAE’s concept dictionary.
  • The steering evaluation combines text prompts with input images and generated responses.
  • Scientific-title examples show generated outputs changing toward white backgrounds, polka dots, or related visual concepts.
  • Intervention strengths are categorized as 0, medium, or high for neurons associated with polka dots, shipwreck, and rainbow.
  • The evaluation uses ten diverse prompts, including requests for poems, scientific titles, to-do lists, dialogues, and math problems.

C User study

The user study tests whether MS agrees with human judgments of which neuron’s top-activating images are more similar and focused. Alignment is measured across neuron pairs and MS distance groups using two image encoders.

  • Each study question compares two grids containing the 16 highest-activation images for neurons kx and ky.Annotators judge which image set looks more similar and focused on the same thing.
  • Human judgments use three annotators per neuron pair, with the final choice determined by majority vote.
  • MS selects neuron kx when MSkx exceeds MSky, and alignment means the MS decision matches the human decision.
  • The overall alignment score is the fraction of evaluated neuron pairs for which MS and humans agree.The study collected 1,000 pair rankings from 71 Mechanical Turk annotators.
  • The study samples neurons from CLIP ViT-L, CLIP ViT-L SAE variants, and a SigLIP SAE, with groups balanced by MS-distance intervals.
  • For MS distances of 0.0–0.1 through 0.4–0.5, CLIP-based alignment scores were 0.55, 0.81, 0.93, 0.96, and 0.93, while DINOv2-based scores were 0.53, 0.77, 0.92, 0.96, and 0.93.Table A1 groups results by MS distance and reports both embedding choices.
  • The appendix provides example user-study questions in Figures A3, A4, and A5.

D Benchmark

The authors release the user-study data as a benchmark for evaluating neuron monosemanticity and comparing future methods with MS. The release includes neuron pairs, top-activating images, activations, and shared evaluation information.

  • The benchmark is intended to support improved alternatives to MS and direct comparison under the same conditions.The authors anticipate developing improved monosemanticity metrics.
  • The release includes pairs.csv with 1,000 neuron pairs, user preferences, and MS values from DINOv2 ViT-B and CLIP ViT-B.
  • top16_images.csv lists the 16 most activating ImageNet training images for each study neuron.
  • The benchmark provides complete neuron activations on the ImageNet validation set for reproducing the evaluation.

E Additional results on monosemanticity

Wider SAE layers produce a broader range of monosemanticity scores and more neurons above any fixed monosemanticity threshold.

  • Wider SAE layers cover a broader range of monosemanticity scores across neurons.
  • Increasing SAE width raises the number of neurons whose monosemanticity exceeds a given threshold.

E.2 Detailed statistics and more models

Additional experiments show that SAE monosemanticity improvements persist across expansion factors, sparsity settings, model architectures, and evaluation encoders, while wider and less sparse representations yield more monosemantic units.

  • Best monosemanticity scores consistently improve as the SAE expansion factor increases, although average scores remain similar.The largest increase occurs between the original layer and an SAE with expansion factor ε = 1.
  • SAE neurons remain more monosemantic than original model neurons when evaluated with CLIP ViT-B embeddings.This pattern holds for SAEs trained on CLIP ViT-L and SigLIP SoViT-400m activations.
  • Across three combinations of explained and evaluation architectures, SAE neurons are more monosemantic than the original neurons.The evaluated combinations use CLIP ViT-L or SigLIP SoViT-400m neurons with DINOv2 or CLIP ViT-B as the evaluation encoder.
  • Higher expansion factors and lower sparsity help identify more monosemantic units when using CLIP ViT-B for evaluation.The same qualitative patterns appear when monosemanticity is computed with DINOv2 ViT-B.
  • The iNaturalist experiment maps neurons to taxonomy depth using top-16 activating images and pairwise Lowest Common Ancestor depths.SAEs are trained with expansion factor ε = 2 and group sizes matching the first five levels of the species taxonomy.
  • At test time, BatchTopK sparsity is approximated with ReLU(x − γ), so measured sparsity can differ slightly from the training target K = 20.
  • A Matryoshka SAE’s highly activating images support steering experiments on a manually labeled “Pencil” neuron.Top-16 activating images are also used to examine overlap between neurons’ preferred concepts.

H Additional qualitative results

Qualitative examples align higher monosemanticity scores with greater similarity among a neuron’s highest-activating images across both DINOv2 and CLIP evaluation encoders.

  • Higher monosemanticity scores correspond to more similar highest-activating images across neurons.The examples are ordered from high to low MS and use DINOv2 ViT-B as the image encoder.
  • The same score–image-similarity correlation appears when CLIP ViT-B is used as the image encoder.
Loading 2504.02821v3…