PaperlayerView exact revision on arXiv

Source-linked AI summary

Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)

Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio P. Calmon, Himabindu Lakkaraju

arXiv:2402.10376v2cs.LGcs.CV

TL;DR

Dense CLIP embeddings are difficult to interpret, leaving their semantic content hard to quantify. SpLiCE decomposes them into sparse combinations of human-interpretable concepts; the resulting representations retain high downstream performance and support spurious-correlation detection and model editing.

  • Problem

    Dense CLIP representations are difficult to interpret, and how to quantify their semantic content remains unclear.

    Source: §1 · pp. 1–2

  • Method

    SpLiCE uses sparse recovery to transform CLIP embeddings into sparse, human-interpretable concept combinations without training.

    Source: §1 · p. 2

  • Results

    SpLiCE produces sparse, interpretable representations with high downstream performance and is demonstrated in spurious-correlation detection and model editing.

    Source: §1 · p. 2

  • Takeaways & Limitations

    SpLiCE embeddings can serve as post-hoc interpretations of CLIP and can replace dense embeddings for transparency.

    Source: §7 · p. 10

    The vocabulary is limited to one- and two-word concepts drawn from LAION-5B token frequencies, whose biases may omit salient concepts encoded by CLIP.

    Source: §7 · p. 10

Abstract

from arXiv · show

CLIP embeddings have demonstrated remarkable performance across a wide range of multimodal applications. However, these high-dimensional, dense vector representations are not easily interpretable, limiting our understanding of the rich structure of CLIP and its use in downstream applications that require transparency. In this work, we show that the semantic structure of CLIP's latent space can be leveraged to provide interpretability, allowing for the decomposition of representations into semantic concepts. We formulate this problem as one of sparse recovery and propose a novel method, Sparse Linear Concept Embeddings, for transforming CLIP representations into sparse linear combinations of human-interpretable concepts. Distinct from previous work, SpLiCE is task-agnostic and can be used, without training, to explain and even replace traditional dense CLIP representations, maintaining high downstream performance while significantly improving their interpretability. We also demonstrate significant use cases of SpLiCE representations including detecting spurious correlations and model editing.

1 Introduction

Dense CLIP representations perform well but their semantic content is difficult to quantify and interpret. The paper introduces SpLiCE, a task-agnostic method for producing sparse concept representations, and presents downstream results and applications.

1 Introduction

  • The paper frames interpretation as a response to difficulty understanding CLIP’s rich, dense representations and quantifying their semantic content.The introduction notes that interpretable decompositions could help examine tasks, biases, and how downstream predictions are made.Source: §1 · pp. 1–2
  • SpLiCE decomposes dense CLIP embeddings into sparse, human-interpretable concept embeddings without training or task-specific concept datasets.The method leverages CLIP’s semantic structure and is presented as a way to explain or replace dense representations.Source: §1 · p. 2
  • The authors report sparse, interpretable representations with high downstream performance that capture the semantics of their inputs.The stated experiments assess performance on downstream tasks.Source: §1 · p. 2
  • SpLiCE is used to detect a CIFAR100 correlation between the “woman” and “swimwear” concepts, attributed to the prevalence of women in swimwear in the dataset.The paper also presents model editing as a use case.Source: §1 · p. 2

2 Related Work

Prior approaches to interpreting representations use linear-semantic assumptions, concept bottlenecks, mechanistic analysis, or disentanglement. The paper distinguishes SpLiCE by using sparse, nonnegative, overcomplete concepts without training or task-specific concept sets.

Linear Representation Hypothesis

  • The paper draws on linear-representation and additive-concept observations to convert dense CLIP representations into sparse semantic ones.The cited discussion also relates image and text embeddings through linear mappings.Source: §2 · p. 2

Concept Bottlenecks and Attribute Learning

  • Concept bottleneck approaches may require expert concept labels or task-specific concepts, whereas SpLiCE uses an overcomplete dictionary and can analyze unlabeled image datasets.The passage contrasts SpLiCE with methods that use large language models to obtain task-specific concept labels.Source: §2 · pp. 2–3

Mechanistic Interpretability and Disentanglement

  • Mechanistic interpretability and disentanglement methods often rely on labels, manual visualization, or costly searches, while mapping arbitrary latent directions to human meanings can be unreliable.SpLiCE instead decomposes CLIP representations into a predetermined set of concepts.Source: §2 · p. 3

CLIP Interpretability

  • SpLiCE is presented as a sparse, nonnegative, overcomplete, task-agnostic alternative that can explain existing CLIP representations without training.The cited approaches include task-specific sparse decompositions, trained architectures, and methods limited to particular model architectures or requiring labeled text.Source: §2 · p. 3

3 When do Sparse Decompositions Exist?

The section identifies conditions under which CLIP representations may admit sparse semantic decompositions, using a latent-concept account of paired image and text data. It also reports preliminary evidence that CLIP embeddings behave approximately linearly under input composition.

3 When do Sparse Decompositions Exist?

  • The data-generating model separates sparse semantic concepts from nonsemantic variation, with images and text produced from shared latent variables.Each nonnegative coordinate of the latent concept vector represents the prevalence of a concept in the underlying data.Source: §3 · p. 4
  • Sparse decomposition is supported under conditions that concepts are sparse, CLIP captures concepts rather than nonsemantic variation, and its encoders are linear in concept space.The image and text encoders are also assumed to be aligned for a given latent concept and variation.Source: §3 · p. 4
  • The authors present the conditions as sufficient reasoning tools, not a claim that they all hold exactly in practice.They identify the linearity condition as especially critical.Source: §3 · p. 4

Sanity Checking CLIP’s Linearity

  • Across datasets and both modalities, composed-input embeddings are approximately averages of the component embeddings, with fitted weights near 0.5.The authors describe this as preliminary evidence for CLIP linearity in image and language embeddings.Source: §3 · pp. 4–5

4 Method

SpLiCE formulates CLIP decomposition as sparse recovery over a language-based concept dictionary, with human interpretability, concise explanations, and constructive nonnegative weights as design goals. The method builds and aligns the vocabulary, then seeks a sparse reconstruction of each centered CLIP embedding.

4.1 Sparse Nonnegative Concept Decomposition

  • SpLiCE represents CLIP embeddings as sparse, nonnegative linear combinations of concept-dictionary elements, formulated as a sparse-recovery problem.The design goals are interpretable atoms, concise decompositions, and constructive weights; negative weights are avoided because they may lack clear semantic meaning.Source: §4 · p. 5, §4.1 · p. 5
  • The task-agnostic vocabulary consists of frequent one- and two-word phrases from LAION-400m captions, filtered for similarity and selected as 10,000 single-word and 5,000 two-word concepts.The paper says this vocabulary imposes minimal priors from outside curators such as human experts or large language models.Source: §4.1 · pp. 5–6
  • The semantic dictionary closely approximates CLIP on zero-shot classification accuracy, but not on cosine similarity to CLIP embeddings.The figure caption attributes this pattern to capturing semantic but not nonsemantic components.Source: Figure 3 · p. 6
  • Because CLIP image and text embeddings occupy different cones, SpLiCE mean-centers image embeddings and concepts, renormalizes them, then restores the image mean for reconstructed embeddings.The image-cone mean is estimated over MSCOCO; the procedure addresses the modality gap that can prevent nonnegative decomposition.Source: §4.1 · p. 6
  • The optimization seeks the sparsest concept weights whose reconstruction reaches a cosine similarity of at least 1 − ϵ, then relaxes the ℓ0 constraint to an MSE objective with an ℓ1 penalty.The reconstruction is given by ˆzimg = σ(Cw∗+µimg).Source: §4.1 · pp. 6–7

5 Experiments

SpLiCE is evaluated for performance, interpretability, and semantic fidelity, with experiments showing strong semantic-task performance and qualitative analyses illustrating the concepts it recovers.

5.1 Setup

  • Experiments use OpenCLIP ViT-B/32, several image and text datasets, and nonnegative Lasso decompositions with sparse solutions and a LAION-token vocabulary.The reported experiments use an l1 penalty yielding l0 norms of 5–20, typically 0.2–0.3 for most datasets.Source: §5.1 · p. 7

5.2 Sparsity-Performance Tradeoffs

  • SpLiCE retains much of black-box CLIP’s performance and significantly outperforms other dictionaries on zero-shot classification, probing, and retrieval, despite weaker cosine reconstruction.The authors attribute the reconstruction gap to CLIP encoding non-semantic information that semantic-only decompositions do not capture.Source: §5.2 · p. 7

5.3 Ablation Studies

  • Ablations find that modality alignment is needed across all three metrics, while negative weights marginally improve cosine reconstruction but reduce semantic relevance.Replacing the semantic dictionary with the LLM-generated dictionary also worsens zero-shot accuracy and cosine reconstruction.Source: §5.3 · p. 8, Figure 5 · p. 8

5.4 Qualitative Assessment of Decompositions

  • Image decompositions recover concepts beyond object identity, including sign text, geometric perspective, and lighting or color information.Across six MSCOCO examples, the top seven concepts generally describe image semantics and suggest use as zero-shot image tags.Source: §5.4 · pp. 8–9
  • Aggregated concept histograms expose dataset patterns, including digital-clock display cues, monarch-butterfly association with lilac flowers, and retriever traits.Manual inspection found purple flowers in nearly half of monarch-butterfly validation images.Source: Figure 4 · p. 8, §5.4 · p. 9

6 Case Studies and Applications of SpLiCE

The case studies use SpLiCE concepts to identify a dataset stereotype and to edit classifiers by targeting specific attributes while preserving other information.

6 Case Studies and Applications of SpLiCE

  • The case studies cover label-free spurious-correlation detection and debiasing classifier models, with additional examples and a user study reported in the appendix.The reported user study found that users preferred SpLiCE explanations over existing Concept Bottleneck Model-based methods.Source: §6 · p. 9

Discovering Spurious Correlations in CIFAR100

  • In CIFAR100, swimwear and bra concepts are common among woman-class images but not among the top clothing concepts for man-class images; manual inspection confirms the trend.At least 70 of 600 woman-class images show women in bikinis, underclothes, or partially undressed.Source: §6 · p. 9

Model Editing on CelebA Attribute Classifiers

  • On CelebA, removing glasses-related concepts reduces glasses-classification performance while preserving gender classification.The study tests concept removal in both SpLiCE representations and linear probes; the table caption describes surgical removal without impacting gender.Source: §6 · pp. 9–10, Table 2 · p. 10

7 Discussion

The discussion presents SpLiCE as sparse recovery of CLIP information into interpretable concepts, with adjustable performance–interpretability tradeoffs and applications in auditing and model intervention.

7 Discussion

  • CLIP information is approximated by sparse linear combinations of simple semantic concepts, yielding human-interpretable decompositions of dense embeddings.The discussion frames the interpretation problem as sparse recovery.Source: §7 · p. 10
  • SpLiCE can serve as a post-hoc interpretation of CLIP or replace dense embeddings for transparency, with an adjustable interpretability–accuracy tradeoff.The tradeoff lets users choose how much performance to give up for interpretability.Source: §7 · p. 10
  • The discussion presents dataset summarization and auditing spurious correlations and biases as uses of SpLiCE at scale.It also connects interpretability with examining model behavior, implicit biases, and individual-sample mistakes.Source: §7 · p. 10
  • The concept vocabulary is limited to one- and two-word concepts drawn from LAION-5B token frequencies, which may miss salient concepts encoded by CLIP.The authors note LAION-5B’s biases and suggest expanding or learning the vocabulary; practitioners can also add concepts for specific uses.Source: §7 · p. 10

Summary of Appendix Results

  • The appendix covers method details, user evaluation, probing and retrieval, additional zero-shot results, and further case studies and analyses.Its listed topics include distribution-shift monitoring, negative concepts, vocabulary choices, and alternative CLIP architectures.Source: Summary of Appendix Results · p. 16

A Further Details on the Method

The appendix grounds SpLiCE’s sparse word-embedding decomposition in stated assumptions and explains its reconstruction objective and batched ADMM solver. It also reports modality alignment, compute setup, and user preference for explanations.

A.1 When do Sparse Decompositions Exist?

  • Under Assumptions 1–5, CLIP image embeddings can be expressed as sparse linear combinations of text embeddings derived from CLIP word embeddings.The proof connects image and text concept vectors and uses word embeddings as the text concept vectors.Source: §A.1 · pp. 16–17

A.2 Relationship between cosine similarity and MSE optimization.

  • For unit-norm vectors, cosine fitting becomes Euclidean reconstruction; the relaxed solution is normalized afterward to recover a unit-norm embedding.The appendix describes the resulting objective as inner-product maximization with shrinkage and sparsity penalties.Source: §A.2 · p. 17

A.3 ADMM for batched on-device LASSO optimization.

  • Batched GPU ADMM reaches 1e-4 primal and dual tolerances in fewer than 1,000 iterations at batch size 1,024, with accuracy and reconstruction approximately matching LASSO.The reported experiment compares zero-shot accuracy and cosine reconstruction on CIFAR100.Source: §A.3 · p. 17, Figure 6 · p. 18

A.4 Effect of Modality Alignment

  • After mean-centering and normalization, average cosine similarity within and across image and text modalities becomes zero, aligning the modalities.Before this alignment, within-modality similarity averages around 0.3, while cross-modal similarity concentrates around zero.Source: §A.4 · p. 18, Figure 7 · p. 18

A.5 Experimental Details

  • Experiments run on a single A100 GPU; after concept-dictionary embedding, computation can run on a CPU.Source: §A.5 · p. 19

B.1 User Study for Human Interpretability

  • In a 20-image ImageNet study, users significantly preferred SpLiCE over LF-CBM and IP-OMP for image relevance and informativeness, without probes or class labels.Users rated explanations on image relevance, prediction relevance, and informativeness; significance was determined with a one-sample two-sided t-test at p=0.01.Source: §B.1 · p. 19

B.2 Performance of SpLiCE on Probing Tasks

Probing evaluations on CIFAR100 and MIT States compare CLIP-trained probes transferred to SpLiCE with probes trained and evaluated on SpLiCE. SpLiCE representations closely match dense CLIP performance, with a slight CIFAR100 drop for directly trained probes.

B.2 Performance of SpLiCE on Probing Tasks

  • CIFAR100 and MIT States probing compares CLIP-trained probes tested on SpLiCE with probes trained and evaluated on SpLiCE.Source: §B.2 · p. 19, Table 3 · p. 19, Table 4 · p. 20
  • SpLiCE representations closely match the performance of dense CLIP embeddings in probing evaluations.Reported means are over three runs, with standard deviations below 0.005.Source: §B.2 · p. 19
  • On CIFAR100, probes trained directly on SpLiCE representations show a slight performance drop versus CLIP-trained probes tested on SpLiCE.Source: §B.2 · p. 19, Table 3 · p. 19

B.3 SpLiCE Performance on Retrieval Tasks

SpLiCE retrieval is evaluated in both image-to-text and text-to-image directions on MSCOCO subsets. Semantic concept dictionaries outperform all baselines at high decomposition sparsity, with an image-dictionary advantage for text-to-image retrieval above 30 nonzero concepts.

B.3 SpLiCE Performance on Retrieval Tasks

  • Retrieval tests cover image-to-text and text-to-image directions, using MSCOCO subsets of 1,024 samples and recall at k = 1, 5, and 10.Source: §B.3 · p. 20, Figure 9 · p. 20
  • At high decomposition sparsity, semantic concept dictionaries outperform all baselines on retrieval.Source: §B.3 · p. 20
  • For text-to-image retrieval, dictionaries learned over images perform slightly better when decompositions have more than 30 nonzero concepts.Source: §B.3 · p. 20

B.4 Additional Zero-Shot Results

Across additional tests, SpLiCE retains zero-shot performance at higher sparsity and supports analysis of concept distributions, spurious correlations, distribution shifts, and vocabulary choices. These studies also probe the behavior of negative concepts and modality centering.

B.4 Additional Zero-Shot Results

  • At sparsities of 20–35, SpLiCE reconstructions are comparable to unaltered CLIP, and at sparsity 512 they recover baseline CLIP zero-shot accuracy.The reported minor performance drop at human-interpretable sparsities contrasts with complete recovery at the baseline embedding sparsity.Source: Table 5 · p. 20, §B.4 · pp. 20–21, Table 6 · p. 21

B.5–B.6 ImageNet Histograms and Spurious Correlations

  • ImageNet histograms expose class-specific concepts and biases, while CIFAR100 shows desert prevalence as a potentially exploitable spurious cue for camels.Examples include a “doll” concept in Kimono images and greater desert prevalence for camels than kangaroos.Source: §B.5 · p. 21, Figure 10 · p. 21, §B.6 · p. 21

B.7 Additional Case Study: Spurious Correlation Intervention

  • On Waterbirds, the dataset spuriously correlates landbirds with land backgrounds; ablating concept weights to remove that information significantly improves worst-case subgroup performance for waterbirds on land from 0.48 to 0.60.Source: §B.7 · pp. 21–22

B.8 Additional Case Study: Distribution Shift Monitoring

  • For Stanford Cars, decomposed “convertible” and “yellow” concept weights generally track yearly ground-truth prevalence, highlighting the out-of-distribution rise of brightly colored sports cars in the early 2000s.The normalized concept-weight axis has no meaningful unit or scale.Source: §B.8 · p. 22, Figure 12 · p. 23

B.9 Additional Case Study: Distribution Shift Monitoring

  • For Waterbirds, validation and test splits are more similar to each other than to training, where “bamboo” is highly unbalanced across landbird and waterbird classes.The bamboo pattern is mostly balanced across both classes in validation and test.Source: §B.9 · pp. 22–23, Table 8 · p. 23, Table 9 · p. 23

B.10–B.11 Negative Concepts and Image-Mean Centering

  • CLIP places antonyms and “not”-prefixed concepts close in cosine similarity, while image-mean decompositions suggest centering removes modality information without removing discriminative semantics.The decomposed image mean yields generally image-related concepts both with and without centering.Source: §B.10 · p. 23, §B.11 · p. 23, Table 10 · p. 24

B.12 Choice of Concept Vocabulary

  • MSCOCO10k and LAION10k perform almost identically on CIFAR100 metrics, while smaller vocabularies retain cosine reconstruction but underperform on zero-shot classification.The vocabulary comparison repeats zero-shot accuracy and reconstruction cosine similarity experiments.Source: §B.12 · p. 24, Figure 13 · p. 24

B.13 Concept Type Distribution

  • Nouns dominate activated concepts across datasets; the authors hypothesize that some adjective and verb information may be encoded in nouns.They suggest “lemon” can succinctly express “yellow” and “fruit.”Source: §B.13 · p. 24

B.14 Experiments on Alternative CLIP Architecture

The authors test SpLiCE with OpenAI’s ResNet50 CLIP architecture using a 10,000-word vocabulary. Results are similar to those reported for another CLIP architecture, although ResNet50 performs worse overall than OpenCLIP ViT B/32.

B.14 Experiments on Alternative CLIP Architecture

  • With a 10,000-word, one-word-concept vocabulary, OpenAI’s ResNet50 CLIP results are similar to those reported earlier, but its performance is generally worse than OpenCLIP ViT B/32.The experiments assess cosine reconstruction and zero-shot accuracy.Source: §B.14 · p. 24, Figure 15 · p. 25
  • SpLiCE decompositions are mostly composed of nouns across multiple datasets.Source: Figure 14 · p. 25
  • Figure 15 covers cosine similarity between CLIP and SpLiCE embeddings and zero-shot classification performance for OpenAI’s ResNet50 CLIP model.Source: Figure 15 · p. 25
Loading 2402.10376v2…