PaperlayerView exact revision on arXiv

Source-linked AI summary

Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery

Sukrut Rao, Sweta Mahajan, Moritz Böhle, Bernt Schiele

arXiv:2407.14499v2cs.CVcs.AIcs.LG

TL;DR

Task-specific concept bottlenecks rely on concepts that may not be detectable by a feature extractor. DN-CBM instead discovers CLIP concepts with sparse autoencoders, names them, and reuses them for classification; the authors report strong performance and interpretability across datasets.

  • Problem

    CBMs often require task-specific concepts, but it is unclear whether a feature extractor can detect every specified concept or whether the resulting explanations are faithful.

    Source: §1 · p. 2

  • Method

    DN-CBM uses sparse autoencoders to discover CLIP concepts, names them by matching decoder vectors to CLIP text embeddings, and trains linear probes on the concepts.

    Source: §1 · p. 2, §3.2 · p. 7, §3.3 · p. 7

  • Results

    DN-CBM is highly performant across datasets and backbones, almost always outperforming task-optimized baselines despite task-agnostic discovery.

    Source: §5.1 · p. 11

  • Takeaways & Limitations

    A discovered and named concept mapping can be reused across varied downstream datasets to build performant, interpretable CBMs.

    Source: §1 · p. 3, §6 · p. 14

    A finite, single-word vocabulary can leave some SAE nodes uninterpretable or give discovered concepts inaccurate names.

    Source: §4.2 · p. 9

Abstract

from arXiv · show

Concept Bottleneck Models (CBMs) have recently been proposed to address the 'black-box' problem of deep neural networks, by first mapping images to a human-understandable concept space and then linearly combining concepts for classification. Such models typically require first coming up with a set of concepts relevant to the task and then aligning the representations of a feature extractor to map to these concepts. However, even with powerful foundational feature extractors like CLIP, there are no guarantees that the specified concepts are detectable. In this work, we leverage recent advances in mechanistic interpretability and propose a novel CBM approach -- called Discover-then-Name-CBM (DN-CBM) -- that inverts the typical paradigm: instead of pre-selecting concepts based on the downstream classification task, we use sparse autoencoders to first discover concepts learnt by the model, and then name them and train linear probes for classification. Our concept extraction strategy is efficient, since it is agnostic to the downstream task, and uses concepts already known to the model. We perform a comprehensive evaluation across multiple datasets and CLIP architectures and show that our method yields semantically meaningful concepts, assigns appropriate names to them that make them easy to interpret, and yields performant and interpretable CBMs. Code available at https://github.com/neuroexplicit-saar/discover-then-name.

1 Introduction

Deep networks can be difficult to interpret, and CBMs still depend on concepts that may not be detectable by the feature extractor. DN-CBM reverses the usual task-first approach by discovering CLIP concepts, naming them, and reusing them for classification.

  • Post-hoc attribution can be unfaithful, motivating inherently interpretable models that explain predictions through concepts, prototypes, or heatmaps.Source: §1 · p. 1
  • Existing CBMs can rely on task-specific concepts whose detectability and faithfulness are uncertain, despite recent label-free approaches using LLMs and VLMs.Source: §1 · p. 2
  • DN-CBM uses sparse autoencoders to discover CLIP concepts, names them through text-embedding alignment, and reuses the frozen bottleneck across classification tasks.The authors report semantically consistent concepts and performant, interpretable classifiers across ImageNet, Places365, CIFAR10, and CIFAR100.Source: §1 · pp. 2–3

2 Related Work

Related work spans concept-based explanations, concept discovery, language-based interpretation, and CBMs. DN-CBM combines SAE-based feature disentanglement with post-hoc naming and a reusable bottleneck for downstream classification.

  • Concept explanations describe high-level decision concepts unlike attribution heatmaps, while neuron-level discovery can yield polysemantic units that are hard to interpret.SAEs have been used to extract more monosemantic concepts from language models; this work extends that setup to CLIP vision features.Source: §2 · p. 3
  • CLIP text embeddings can name SAE dictionary vectors directly, yielding interpretable concepts without the LLM-based mapping used by other language-explanation methods.Source: §2 · p. 4
  • Earlier CBMs may require labelled concepts or task-specific LLM queries, whereas DN-CBM discovers concepts independently and reuses its bottleneck across datasets.Unlike [27], it needs no external image-text annotations and can discover concepts on data separate from downstream tasks.Source: §2 · p. 4
  • For CLIP, DN-CBM differs from by discovering concepts before naming them, and from concurrent SAE analysis by using dictionary vectors to name concepts and build CBMs.Source: §2 · pp. 4–5

3 Constructing CBMs via Automated Concept Discovery

DN-CBM discovers sparse CLIP features with an SAE, names them by matching decoder vectors to CLIP text embeddings, and trains linear probes over the resulting concept bottleneck.

3 Constructing CBMs via Automated Concept Discovery

  • The overall pipeline discovers CLIP concepts, names them through text embeddings, and uses the named concepts as a bottleneck for downstream classification.Source: §3 · p. 5

3.1 Extracting Concepts Learned by the Model

  • SAEs represent CLIP features in a higher-dimensional sparse space, using reconstruction and L1 sparsity objectives on an unlabeled discovery dataset.Sparsity does not guarantee human interpretability, but the authors find it often occurs; SAEs can represent more features than neurons, which may help address polysemanticity.Source: §3.1 · p. 6

3.2 Automated Concept Naming

  • Each SAE decoder dictionary vector is named by matching it to the closest CLIP text embedding from a large English-word vocabulary.This relies on dictionary vectors resembling embeddings of words associated with the corresponding images; the authors describe the naming as equivalent to reconstructing a feature with only that concept present.Source: §3.2 · pp. 6–7

3.3 Constructing Concept Bottleneck Models

  • A linear probe trained on sparse SAE activations forms the classifier, using cross-entropy and an L1 penalty on probe weights.The feature extractor, discovery dataset, and naming vocabulary can be chosen independently.Source: §3.3 · p. 7

4 Evaluation of Concept Discovery and Naming

Using CLIP ResNet-50 features, CC3M-trained sparse autoencoders, and a 20k-word vocabulary, the evaluation examines task-agnostic concept quality, naming, vocabulary granularity, and concept clustering. It finds that discovered concepts are often semantically consistent, while alignment and vocabulary granularity relate to interpretability and naming accuracy.

4.1 Task-Agnosticity and Accuracy of Concepts

  • Named concepts activated semantically consistent images across four datasets, including “asleep” images spanning different species, despite those datasets being unused for extraction and naming.Qualitative examples included low-level, object- and scene-specific, and higher-level concepts.Source: §4.1 · p. 8, Figure 3 · p. 8
  • SAE concepts were significantly more human-interpretable than CLIP features, and SAE names generally scored higher in accuracy; consistency decreased as alignment decreased.The user study assessed concepts with high, intermediate, and low alignment; some low-alignment nodes did not correspond to a consistent human-interpretable concept.Source: §4.1 · p. 9, Figure 4 · p. 10
  • 18.3 Jaccard index for the SAE concepts versus 22.0 for CLIP retrieval; reported 15.7 under a similar setting.The SAE was not optimized to learn dataset-specific concepts.Source: §4.1 · p. 9

4.2 Impact of Vocabulary on Concept Name Granularity

  • Adding fine-grained vocabulary names distinguished concepts otherwise assigned the same coarse label, while removing a name led to worse labels; vocabulary granularity can affect naming accuracy.Examples include “arch bridge” and “suspension bridge” when added, versus “prague” and “lisbon” when “bridge” was removed.Source: §4.2 · pp. 9–10, Figure 5 · p. 10

4.3 Clustering Concept Vectors

  • K-Means clusters of Places365 concept activations grouped semantically similar concepts and images, including farming-related examples.Clusters were represented by their centroid’s strongest concepts and randomly selected assigned images.Source: §4.3 · p. 10, Figure 6 · p. 11

5 Evaluation of DN-CBM

DN-CBM is evaluated for classification, interpretability, and interventions across datasets and backbones. It often outperforms prior CBMs, gives semantically relevant explanations, and responds to concept interventions on Waterbirds.

5.1 Classification Performance

  • DN-CBM performs competitively and often outperforms prior work across four datasets, despite using a common concept set across datasets.The evaluation compares classification accuracy with CLIP ResNet-50 and ViT-B/16 feature extractors.Source: §5.1 · p. 11, Table 1 · p. 12

5.2 Interpretability of DN-CBM

  • Image-level explanations use diverse, intuitive concepts, while class-level explanations are semantically consistent and resemble baseline explanations despite task-agnostic concept extraction.Concepts can reflect scene objects, similar features, higher-level concepts, or class-associated locations; the most influential concepts can vary with image contents.Source: §5.2 · p. 12, Figure 8 · p. 13, Figure 9 · p. 13

5.3 Effectiveness of Concept Interventions

  • Keeping bird-related concepts improves overall and worst-group Waterbirds accuracy, while removing them produces a large accuracy drop.Keeping only bird-related concepts causes a small drop in the other groups; removing them does not adversely affect training-group performance.Source: §5.3 · p. 14, Table 2 · p. 14

6 Conclusion

The conclusion presents DN-CBM as a task-agnostic method for discovering and naming CLIP concepts, then using them for classification. It reports meaningful concepts and performant, interpretable CBMs, and identifies scaling the SAE as future work.

  • DN-CBM uses sparse autoencoders to discover and name CLIP concepts, then uses the learned representations as a bottleneck for linear classification.Source: §6 · p. 14
  • The approach yields semantically consistent concepts with appropriate names and performant, interpretable CBMs across downstream datasets despite task-agnostic extraction.Source: §6 · p. 14
  • A larger-dataset, higher-dimensional foundational SAE is proposed as future work, particularly for fine-grained classification.Source: §6 · p. 14

A Limitations and Broader Impact

The discussion reports that CC3M-trained SAEs yield coarse concepts and do not capture highly fine-grained concepts. It also identifies spurious correlations and concept-naming vocabulary as areas for future work.

B Implementation Details

The appendix section says it provides additional details for the evaluations in Sections 4 and 5.

B.1 Training SAEs and DN-CBM

The authors train SAEs on CC3M CLIP features, then train sparse linear probes on the discovered concepts for downstream classification. For SUNAttributes, they evaluate concept accuracy by grouping and thresholding discovered concepts against ground-truth attributes.

B.1 Training SAEs and DN-CBM

  • SAEs are trained on pooled CLIP features from CC3M, and sparse linear probes are trained on their concept representations for classification.The authors use ResNet-50, ViT-B/16, and ViT-L/14 feature extractors; probes use an L1 constraint on their weights.Source: §B.1 · p. 21

B.2 Concept Accuracy on SUNAttributes

  • Increasing the SAE L1 sparsity parameter produces sparser concept representations but increases reconstruction error.The figure compares reconstruction error with the number of concepts used across sparsity-parameter settings.Source: Figure B1 · p. 22
  • SUNAttributes concept accuracy is measured by averaging per-image Jaccard agreement between predicted and ground-truth concepts.Because the SAE latent space exceeds the 102 ground-truth concepts, same-name nodes are clustered and filtered, then assigned the maximum constituent concept strength and a heldout-set threshold.Source: §B.2 · p. 22

B.3 User Study

The user study assesses semantic consistency and naming accuracy for SAE nodes against CLIP-Dissect nodes. The intervention evaluation groups each class’s five concepts as bird or non-bird concepts.

B.3 User Study

  • The user study compares named SAE nodes with CLIP-Dissect-named CLIP features, sampling nodes across high, intermediate, and low name-to-feature alignment.Participants rated whether activating images shared an interpretable concept and whether its assigned name described that concept; 22 people responded.Source: §B.3 · pp. 22–23

B.4 Applying Interventions on DN-CBM

  • For intervention evaluation, each class’s five concepts are manually classified as bird or non-bird concepts.Table B1 lists the concepts used for these interventions.Source: Table B1 · p. 25, §B.4 · p. 25

C Additional Quantitative Results

The appendix adds classification results for additional backbones and examines the accuracy–sparsity trade-off in DN-CBM explanations.

C Additional Quantitative Results

  • Additional quantitative results cover DN-CBM performance on more backbones and the trade-off between classification accuracy and explanation sparsity.The appendix directs readers to C.1 for additional-backbone performance and C.2 for the accuracy–sparsity analysis.Source: C Additional Quantitative Results · p. 26

C.1 DN-CBM Classification Performance

Across four datasets and three CLIP backbones, DN-CBM outperforms the reported baselines and nearly bridges the accuracy gap with linear probes. Its explanations can be made highly sparse with only a small accuracy reduction; the appendix also provides further qualitative examples.

C.1 DN-CBM Classification Performance

  • DN-CBM outperforms all baselines and almost completely bridges the accuracy gap with linear probes.The reported classification tables cover ImageNet, Places365, CIFAR10, and CIFAR100 across CLIP ResNet-50, ViT-B/16, and ViT-L/14.Source: §C.1 · p. 26, Table C1 · p. 26, Table C2 · p. 27
  • Task-specific SAE selection slightly improves on task-agnostic DN-CBM, but the performance difference is very small.The task-specific variant selects an SAE configuration for each dataset using heldout validation accuracy.Source: §C.1 · p. 26

C.2 Sparsity of DN-CBM Explanations

  • DN-CBM can use very few concepts per decision on average while sacrificing only a small amount of accuracy.The analysis measures sparsity as the average number of concepts needed to reach 0.9 of the original logit value.Source: §C.2 · p. 28, Figure C1 · p. 28

D Additional Qualitative Results

  • The appendix provides additional qualitative examples of task-agnostic discovered concepts, concept-strength clustering, and local and global explanations.It also discusses findings associated with the local and global explanation examples.Source: D Additional Qualitative Results · p. 29

D.1 Task Agnosticity of Concepts

Across three CLIP backbones, discovered concepts are semantically consistent with their names despite visually diverse activating images; additional qualitative results show coherent clusters and useful local and class-wise explanations.

D.1 Task Agnosticity of Concepts

  • Across backbones, discovered concepts align with their names and span simple-to-complex concepts while activating on visually diverse but semantically related images.Examples range from colors to higher-level concepts such as conversation.Source: §D.1 · p. 29, Figure D1 · p. 30, Figure D2 · p. 31

D.2 Clustering Concept Strength Vectors

  • Clustering SAE concept strengths produces visually similar image groups and highly semantically consistent clusters on ImageNet.The illustrated clusters include distinct groups such as eagles, knitted garments, ships, infant rabbits, and infant dogs.Source: §D.2 · p. 29, Figure D4 · p. 33

D.3 Local Explanations from DN-CBM

  • Local explanations typically describe images through diverse contributing concepts and can help explain misclassified decisions.Examples include misclassifications involving visually similar classes and concepts that appear spuriously correlated.Source: §D.3 · p. 34, Figure D5 · p. 35, Figure D6 · p. 36

D.4 Global Explanations from DN-CBM

  • Across backbones, top-contributing concepts provide relevant class-wise explanations on Places365.The examples include concepts describing visible objects, related places, and associations with a class.Source: §D.4 · p. 34, Figure D8 · p. 38

Related papers

Loading 2407.14499v2…