PaperlayerView exact revision on arXiv

Source-linked AI summary

Label-Free Concept Bottleneck Models

Tuomas Oikarinen, Subhro Das, Lam M. Nguyen, Tsui-Wei Weng

arXiv:2304.06129v2cs.LGcs.CV

TL;DR

Existing concept bottleneck models require labeled concept data and can lose accuracy relative to standard neural networks, limiting their practicality. Label-free CBM transforms neural-network backbones into interpretable CBMs using foundation models and automated concept and projection procedures. It is reported to preserve high accuracy, scale to ImageNet, and provide interpretable decision rules and explanations, while errors can arise from missing concepts or incorrect concept activations.

  • Problem

    Existing CBMs require time-consuming labeled concept data and often have lower accuracy than standard neural networks, especially on complex datasets.

  • Method

    Label-free CBM uses foundation models to transform a neural-network backbone into an interpretable CBM without labeled concept data, including automated concept-set creation and projection learning.

  • Results

    Label-free CBM creates high-performing CBMs across five datasets, including a 72% top-1 accuracy ImageNet model, while producing interpretable decision rules and explanations.

  • Takeaways & Limitations

    The framework is scalable to ImageNet, efficient for very large datasets, automated for new datasets, and supports understanding and manual editing of trained models.

  • Takeaways & Limitations

    Some important fine-grained concepts may be absent from the concept bottleneck layer, while incorrect concept activations can cause incorrect predictions.

Abstract

from arXiv · show

Concept bottleneck models (CBM) are a popular way of creating more interpretable neural networks by having hidden layer neurons correspond to human-understandable concepts. However, existing CBMs and their variants have two crucial limitations: first, they need to collect labeled data for each of the predefined concepts, which is time consuming and labor intensive; second, the accuracy of a CBM is often significantly lower than that of a standard neural network, especially on more complex datasets. This poor performance creates a barrier for adopting CBMs in practical real world applications. Motivated by these challenges, we propose Label-free CBM which is a novel framework to transform any neural network into an interpretable CBM without labeled concept data, while retaining a high accuracy. Our Label-free CBM has many advantages, it is: scalable - we present the first CBM scaled to ImageNet, efficient - creating a CBM takes only a few hours even for very large datasets, and automated - training it for a new dataset requires minimal human effort. Our code is available at https://github.com/Trustworthy-ML-Lab/Label-free-CBM. Finally, in Appendix B we conduct a large scale user evaluation of the interpretability of our method.

1 INTRODUCTION

Deep neural networks achieve strong performance but are difficult to interpret, motivating Label-free CBM, which converts neural networks into interpretable CBMs without labeled concept data. The framework is designed to preserve accuracy while improving scalability, efficiency, and automation.

  • 1 INTRODUCTION: Deep neural networks perform strongly across tasks but are difficult to understand because of their complex, deep structures.Interpretability matters for calibrated trust and knowing when models should be trusted.
  • 1 INTRODUCTION: Concept Bottleneck Models place human-understandable concepts in an intermediate layer, making final decisions linear functions of interpretable concepts.CBMs have been used for applications including model debugging and human intervention.
  • 1 INTRODUCTION: Label-free CBM transforms any neural network into an interpretable CBM without labeled concept data while preserving accuracy comparable to the original network.The framework leverages foundation models to address the limitations of existing CBMs.
  • 1 INTRODUCTION: The method is presented as scalable, with the first CBM reported to scale to ImageNet.This expands CBM application beyond smaller datasets.
  • 1 INTRODUCTION: Creating a CBM takes only a few hours even for very large datasets.This supports practical deployment on datasets that would otherwise make CBM construction costly.
  • 1 INTRODUCTION: Training Label-free CBM for a new dataset requires minimal human effort.The framework is intended to automate CBM construction rather than depend on extensive expert labeling.

2 RELATED WORK

Prior interpretability methods modify outputs, approximate decisions, or rely on concept annotations and specialized backbones. Label-free CBM is positioned against these limitations by targeting interpretable concept representations without those requirements.

  • 2 RELATED WORK: Post-hoc explanations such as LIME and SHAP use local approximations and pixel-level importance, which may be inaccurate or require subjective human analysis.Their explanations focus on individual model decisions rather than an interpretable concept bottleneck.
  • 2 RELATED WORK: Sparse final layers and neural-backed decision trees improve final-layer interpretability, but preceding features can remain uninterpretable.Other approaches include Concept Whitening and Concept Embedding Models.
  • 2 RELATED WORK: Existing CBMs use concept bottleneck neurons corresponding to human-interpretable concepts and support test-time intervention and debugging.Post-Hoc CBM reduces training cost but still has important limitations.
  • 2 RELATED WORK: Post-Hoc CBM may require annotated concept data with TCAV, is restricted to CLIP image-encoder backbones when using CLIP, and can lose substantial accuracy without uninterpretable residual layers.These constraints motivate methods that avoid concept annotations and preserve full interpretability.
  • 2 RELATED WORK: Table 1 compares CBM, IBD, and four P-CBM variants, distinguishing hybrid models with uninterpretable residual terms and models using CLIP concepts.The table also marks methods that could theoretically extend to ImageNet but have not been tested there.
  • 2 RELATED WORK: CLIP-Dissect measures how closely hidden neurons represent concepts without concept annotations, providing an optimization target for learning an interpretable projection.This connects neuron interpretation with Label-free CBM’s projection-learning procedure.

3 LABEL-FREE CBM: A NEW FRAMEWORK TO BUILD CBM

Label-free CBM transforms a neural-network backbone into an interpretable concept bottleneck model without labeled concept data through an automated four-step pipeline. It generates and filters concepts, learns an interpretable projection using CLIP-based signals, and trains a sparse final predictor.

  • The framework transforms a neural-network backbone into an interpretable CBM without concept labels through four automated steps.The steps create and filter concepts, compute backbone and concept embeddings, learn projection weights, and train the final layer.
  • Step 1: Concept set creation and filtering: GPT-3 generates initial concepts from class names using prompts about features, contexts, and superclasses, with shared few-shot examples across datasets.The resulting concepts are combined across classes and prompts before filtering.
  • Step 1: Concept set creation and filtering: Concept filtering removes overly long, class-similar, duplicate, absent, and inaccurately projectable concepts to simplify and improve the bottleneck.The filters use character length, embedding similarity, CLIP activation on training data, and projection interpretability.
  • Steps 2–3: Learning the concept bottleneck layer: The method learns projection weights without concept labels by matching projected neuron activation patterns to CLIP-derived concept activations.The target concepts form C, and the CLIP concept activation matrix uses image-text embedding products over the training data.
  • Steps 2–3: Learning the concept bottleneck layer: Cos cubed similarity compares normalized, element-wise cubed activation vectors, increasing sensitivity to highly activating inputs during projection optimization.The projection is optimized with Adam and early stopping based on validation similarity.
  • Step 4: Learning the sparse final layer: The final predictor is trained as a sparse linear model with an elastic net objective after fixing the backbone and concept projection.The implementation targets 25 to 35 nonzero weights per output class to retain interpretable decisions and accuracy.

4 EXPERIMENT RESULTS

The experiments evaluate Label-free CBM across five diverse image-classification datasets and show that it preserves accuracy while enabling interpretable global and individual decisions. The method reaches 72% top-1 accuracy on ImageNet, outperforms Post-hoc CBM where compared, and performs similarly to sparse standard models.

  • Datasets and setup: Experiments evaluate Label-free CBM on CIFAR-10, CIFAR-100, CUB, Places365, and ImageNet, spanning general, fine-grained, and scene-recognition tasks.The datasets vary substantially in task type and size.
  • Result (I): Accuracy: 72% top-1 accuracy on ImageNet demonstrates that Label-free CBM scales to a large classification dataset with little accuracy loss.The method creates sparse-final-layer CBMs across all five evaluated datasets.
  • Result (I): Accuracy: Label-free CBM significantly outperforms Post-hoc CBM on evaluated datasets and performs similarly to sparse standard models.On CUB, it also surpasses Post-hoc CBM despite using a GPT-3-derived concept set rather than expert-provided concepts.
  • Result (II): Explainable decision rules: Figure 3 represents concept-to-class weights as Sankey-line widths for pairs of similar classes, with negative weights denoting NOT concepts.Only weights with absolute value greater than 0.05 are shown, while typical largest weights range from 0.5 to 1.
  • Result (III): Explainable individual decisions: Because decisions are linear functions of normalized interpretable features, the model provides simple explanations for individual predictions.Concept activations are normalized to mean 0 and standard deviation 1 on the training data before contributions are assessed.

5 CASE STUDY: MANUALLY IMPROVING AN IMAGENET MODEL

The ImageNet case study categorizes model errors and shows that inspecting interpretable concepts enables targeted final-layer edits that improve validation accuracy.

  • 5.1 TYPES OF MODEL ERRORS: The study identifies four error types: ambiguous labels, missing concepts, incorrect activations, and incorrect final-layer weights.Each error type suggests a different response, while the study focuses mainly on correcting final-layer weights.
  • 5.1 TYPES OF MODEL ERRORS: Type 4 errors occur when concept activations are correct but the final-layer weights still produce an erroneous prediction.Because the final layer is interpretable, these errors are the main target of the manual-editing procedure.
  • 5.2 EDITING FINAL LAYER WEIGHTS: The editing procedure visualizes incorrect predictions, selects a highly activated concept favoring the ground-truth class, and adjusts its weights for the ground-truth and predicted classes.The weight changes use WF[gt,concept] ← WF[gt,concept] + ∆w and WF[pred,concept] ← WF[pred,concept] −∆w, with validation checks for broader effects.
  • 5.2 EDITING FINAL LAYER WEIGHTS: For a cricket insect prediction changed to grasshopper, the authors selected “a green color” because it distinguished the ground-truth class while limiting broader impact.The concept was highly activated and relevant to the desired correction.
  • 5.2 EDITING FINAL LAYER WEIGHTS: After five edits, validation accuracy rose from 71.98% to 72.02%, correcting 38 predictions while turning 17 incorrect.The edits changed only 10 of 4.5 million final-layer weights and improved accuracy on the affected 10-class subset by 4.2%.

6 CONCLUSION

The paper concludes that Label-free CBM provides an automated, scalable route to high-performing concept bottleneck models, with interpretable decisions and manual editing benefits. It also reports crowdsourced evidence that LF-CBM is more interpretable than standard models.

  • 6 CONCLUSION: Label-free CBM is presented as a fully automated and scalable method that produces the first high-performing CBM on ImageNet.The framework is intended to generate concept bottleneck models without the limitations of existing approaches.
  • 6 CONCLUSION: The models support both global decision rules and explanations for individual predictions through interpretable concepts.The conclusion connects these explanations to the model’s concept-based decision structure.
  • 6 CONCLUSION: Manual final-layer weight edits can improve accuracy in a trained ImageNet model using the understanding provided by the concept bottleneck representation.The paper presents editing as an application of the model’s interpretability.
  • 6 CONCLUSION: A large-scale crowdsourced evaluation reports that LF-CBM is more interpretable than standard models according to two metrics.The evaluation is described in Appendix B.

A.2 LIMITATIONS

The method’s concept generation can omit important domain knowledge, and its performance depends on whether CLIP works well in the target domain. These limitations make human collaboration or alternative CBMs preferable in some settings.

  • A.2 LIMITATIONS: GPT-3 concept generation is stochastic and may fail to produce important concepts for detecting some classes.The authors report robustness to small concept-set changes, but automatic concept generation can still lack required domain knowledge.
  • A.2 LIMITATIONS: Automatically generated concept sets may omit domain-specific concepts, so collaboration with a human expert may be preferable.This limitation concerns the domain knowledge needed to include important concepts.
  • A.2 LIMITATIONS: The method works best where CLIP performs well and may perform less effectively on small, specialized datasets such as medical datasets.For such datasets, the authors favor methods that can leverage available concept labels, including original CBMs or P-CBM.

A.4 ABLATION: EFFECT OF CONCEPT FILTERS

Concept filters are designed primarily to improve interpretability and efficiency rather than accuracy. Ablations show little accuracy sensitivity, while filtered models produce cleaner, more meaningful decision rules.

  • A.4 ABLATION: EFFECT OF CONCEPT FILTERS: The filters aim to improve interpretability and reduce computational complexity by reducing the number of concepts.They are not designed to improve model accuracy.
  • A.4 ABLATION: EFFECT OF CONCEPT FILTERS: ImageNet accuracy remained constant across filter ablations, while CIFAR-10 accuracy slightly increased with fewer filters.The authors could not train an ImageNet model without any filters because it did not become sufficiently sparse.
  • A.4 ABLATION: EFFECT OF CONCEPT FILTERS: The CIFAR-10 filtering procedure starts from 177 GPT-3-generated concepts and removes concepts that are too long or too similar to output classes.These steps seek simple, non-convoluted concepts and more informative explanations.
  • A.4 ABLATION: EFFECT OF CONCEPT FILTERS: Filter cutoffs are mainly chosen through trial and error for useful, explainable models, with most cutoffs independent of the dataset.The cutoff for images being present in the dataset is the stated dataset-dependent exception.
  • A.5 FURTHER DISCUSSION AND EXAMPLE OF CONCEPT FILTERING: With filters, the CIFAR-10 automobile model assigns its largest weights to “four wheels” and “a steering wheel,” producing a reasonable decision rule.Without filters, duplicate class-like concepts and an unrelated “bed” concept make the rule less informative.

A.6 ABLATION: EFFECT OF INITIAL CONCEPT SET

The ablation compares GPT-3 and ConceptNet for generating the initial concept set, noting that ConceptNet works best for common single-word phrases while many CUB class names are more complex.

  • The experiment compares the proposed GPT-3 concept-set generator with ConceptNet as an alternative for creating initial concepts.The comparison follows the ConceptNet-based approach proposed by Yuksekgonul et al. (2022).

A.7 ABLATION: LF-CBM WITHOUT SPARSITY CONSTRAINT

Removing sparsity constraints improves accuracy but substantially weakens interpretability because the most influential concepts explain little of the decision. Additional passages document model editing and interpretability evaluations.

  • A.7 ABLATION: LF-CBM WITHOUT SPARSITY CONSTRAINT: Removing sparsity constraints greatly improves accuracy, except on CUB-200 where the dense model begins to overfit.
  • A.7 ABLATION: LF-CBM WITHOUT SPARSITY CONSTRAINT: Without sparsity, the ten most highly contributing concepts explain almost none of the decision.
  • A.8 CONSISTENCY OF CONCEPT SET GENERATION: Regenerating the ImageNet concept set produced 4380 rather than 4523 concepts, while accuracy changed only from 71.95% to 71.89%.The concepts and weights could nevertheless differ substantially while remaining relevant.
  • A.9 MORE DETAILS ON MODEL EDITING PROCEDURE: Weight editing increased overall accuracy from 71.98% to 72.02% after edits to only five images.The edits affected weights for 10 of 1000 classes, corresponding to a 4% accuracy boost on affected classes.
  • A.9 MORE DETAILS ON MODEL EDITING PROCEDURE: Weight edits may affect other predictions, so each edit should be checked on a validation dataset before applying it.
  • Additional interpretability results: Visualized final-layer weights for Places365 and ImageNet assigned large weights to concepts relevant to the output classes.

B.1 TASK 1: DO NEURONS IN CBL CORRESPOND TO THEIR TARGET CONCEPT?

The first user study tests whether Concept Bottleneck Layer neurons match their target concepts by comparing ImageNet LF-CBM neurons with neurons from its standard ResNet-50 backbone.

  • Setup: The study evaluated all 4505 ImageNet LF-CBM bottleneck neurons using three ratings per neuron, based on their ten most highly activating validation images.
  • Results: LF-CBM bottleneck neurons averaged 3.91 versus 3.65 for standard neurons, indicating higher judged interpretability.
  • Results: 76.4% of raters agreed or strongly agreed for LF-CBM neurons, compared with 65.2% for standard neurons.
  • Results: Author evaluations likewise favored LF-CBM neurons, averaging 3.66 versus 3.30 for ResNet-50 neurons.

B.2 TASK 2: HOW REASONABLE ARE OUR EXPLANATIONS FOR A SINGLE DECISION?

The second user study compares the reasonableness of LF-CBM and baseline explanations for individual correct predictions. Users generally preferred LF-CBM explanations, though the evaluation did not measure explanation comprehensiveness.

  • Setup: Users compared explanations showing the five most contributing positive concepts for LF-CBM and baseline models on inputs all models classified correctly.The study used 350 random images, three raters per comparison, and 3150 total comparisons.
  • Results: Users preferred LF-CBM explanations over standard dense and sparse models in 64.2% and 65.2% of unfiltered comparisons, respectively.
  • Results: After filtering inconsistent workers, preference for LF-CBM rose to 74.4% over standard dense and 78.8% over standard sparse explanations.
  • Results: The most common reasons for preferring an explanation were greater relevance to the image and prediction, selected 80% and 60% of the time.
Loading 2304.06129v2…